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Abstract 

In  the  context  of  close  ground  combat,  the  perception  of  Battle  Damage  Assess¬ 
ment  (BDA)  is  closely  linked  with  a  soldier’s  engagement  decisions  and  has  significant 
effects  on  the  battlefield.  Perceived  BDA  is  also  one  of  the  most  complex  and  uncertain 
processes  facing  the  soldier  in  live  combat.  As  a  result,  the  modeling  and  simulation 
community  has  yet  to  adequately  model  the  perceived  BDA  process  in  combat  mod¬ 
els.  This  research  effort  examines  the  BDA  process  from  a  perception  standpoint  and 
proposes  a  methodology  to  collect  the  pertinent  data  and  model  this  perception  in  the 
Army’s  current  force-on-force  model,  CASTFOREM.  A  subject  matter  expert  survey 
design  and  a  method  to  model  the  BDA  process  as  a  Discrete  Time  Markov  Chain 
are  proposed.  Bayesian  inference  is  used  to  update  probability  distributions  at  each 
time  step  considering  the  situational  parameters  available  to  the  soldier  at  the  time  of 
an  assessment.  Comparisons  between  the  known  simulation  distributions  and  those 
developed  from  simulated  survey  responses  suggest  an  adequate  number  of  subject 
matter  experts  to  be  polled. 
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BDA  ENHANCEMENT  METHODOLOGY 

USING 

SITUATIONAL  PARAMETER  ADJUSTMENTS 

I.  Introduction 

1.1  Background 

The  research  presented  in  this  thesis  is  sponsored  by  the  Training  and  Doctrine 
Command  (TRADOC)  Analysis  Center  (TRAC).  TRAC  performs  analysis  to  help 
shape  the  future  of  the  Army  and  Department  of  Defense  (DoD)  over  a  five  to  fifteen 
year  horizon,  focusing  on  such  areas  as  analysis  of  alternatives,  organization  and  op¬ 
erations,  and  modeling  and  simulation  (M&S)  development  and  maintenance.  During 
an  analysis  of  alternatives,  TRAC  found  their  representation  of  battle  damage  assess¬ 
ment  (BDA)  in  force-on-force  models  inadequate  and  set  to  initiate  a  BDA  Project. 
The  primary  goal  of  the  TRAC  BDA  project  is  to  effectively  represent  the  BDA  pro¬ 
cess  in  combat  models.  TRAC  is  also  responsible  for  the  Army’s  main  force-on-force 
model,  the  Combined  Arms  Support  Task  Force  Evaluation  Model  (CASTFOREM), 
and  wished  to  improve  it  by  employing  the  Air  Force  Institute  of  Technology  (AFIT) 
to  develop  a  new  BDA  methodology.  The  goal  of  this  research  is  to  provide  a  flexible 
methodology  which  TRAC  can  implement  not  only  in  CASTFOREM,  but  also  other 
models  such  as  CombatXXI,  the  future  replacement  for  CASTFOREM. 

This  chapter  will  present  an  introduction  to  BDA  concepts  followed  by  a  dis¬ 
cussion  of  how  CASTFOREM  models  BDA  currently.  Subsequently,  the  problem  at 
hand  will  be  formulated  and  scoped  for  this  research. 

1.2  BDA  Concepts 

Most  of  the  current  military  doctrine  regards  BDA  as  an  air-centric  warfare 
component.  However,  this  research  specifically  deals  with  BDA  using  ground  weapon 
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systems  in  real  time  (i.e.  during  an  engagement).  Accordingly,  much  of  the  doctrine 
applies  to  the  problem  at  hand  only  in  a  broad  sense.  The  DoD  defines  BDA  as 


The  timely  and  accurate  estimate  of  damage  resulting  from  the  applica¬ 
tion  of  military  force,  either  lethal  or  nonlethal,  against  a  predetermined 
objective.  Battle  damage  assessment  can  be  applied  to  the  employment  of 
all  types  of  weapon  systems  (air,  ground,  naval,  and  special  forces  weapon 
systems)  throughout  the  range  of  military  operations...  [12] 

BDA  is  part  of  the  combat  assessment  (CA)  process  which  is,  in  turn,  part  of 
the  joint  targeting  cycle  shown  in  figure  1.1.  BDA,  munitions  effectiveness  assessment 


Assess 


Critical  assessments  of  strike 
effects. 

Determines  battle  damage, 
reattack  recommendations, 
and  munitions  performance. 


Mission  Planning 
and  Force  Execution 

Detailed  operational  planning 
and  execution  of  operations. 


Deliver 


Decide 


Commander's  Objectives, 
Guidance,  and  Intent 

Identifies  military  targets/objectives 
in  support  of  national  objectives, 
and  key  limitations  on  operations. 
Drives  subsequent  phases 
of  the  targeting  cycle. 


Commander’s  Decision 
and  Force  Assignment 

1  Validates  that  the  target  supports 
commander's  objectives. 

1  Provides  formal  command 
approval  for  the  strike. 


Target 

Development 

1  Relies  on  the  interaction  of 
intelligence  assets  to  identify 
target  characteristics  and 
validate  that  the  target  is  legal. 


Capabilities 

Analysis 

1  Assesses  the  availability  of 
strike  assets  and  forces. 

1  Ensures  achievement  of  desired 
outcomes  and  objectives. 


Detect 


Source:  DOD. 


Figure  1.1:  Joint  Targeting  Cycle  [3] 


(MEA),  and  future  target  nominations  or  re-attack  recommendations  make  up  CA. 
BDA  occurs  in  three  phases;  physical  damage  assessment  (Phase  I),  functional  damage 
assessment  (Phase  II),  and  target  system  assessment  (phase  III).  Each  phase  extends 
the  information  from  the  last  phase,  to  make  a  further  determination  of  battle  damage 
and  its  effects.  The  DoD  [12]  defines  the  three  phases  of  BDA  as  follows: 
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Phase  I  -  Physical  Damage  Assessment  An  estimation  of  the  quantitative  ex¬ 
tent  of  physical  damage  (through  the  application  of  military  force)  to  a  target 
element,  based  on  observed  or  interpreted  damage. 

Phase  II  -  Functional  Damage  Assessment  A  continuation  of  phase  I  assess¬ 
ments.  The  estimate  of  the  effect  physical  damage  has  on  a  target’s  functional 
or  operational  capability. 

Phase  III  -  Target  System  Assessment  An  aggregation  of  phase  II  effects  result¬ 
ing  in  a  judgement  of  theater- wide  weapons  system  capabilities  and  a  determi¬ 
nation  of  the  enemy’s  ability  to  wage  war. 

The  focus  of  this  study  is  on  Phase  I  and  II  BDA. 

1.2.1  Kill  Types.  The  Joint  Munitions  Effectiveness  Manual  (JMEM)  de- 
hnes  a  list  of  45  kill  types  in  combat  operations.  A  list  of  some  possible  kill  states  is 
presented  in  Table  1.1. 


Table  1.1:  JMEM  Kill  Types 


Aircraft  Control  kill 
Catastrophic  kill  (K-kill) 

Catastrophic  on  Ground  kill  (COG-kill) 
Communications  kill 
Data  Processing  kill 
Expedient  Interdiction  kill 
Firepower  kill  (F-kill) 

Incapacitation  kill 

Long  Range  Sur.-to-Air  Firepower  kill 
Mission  Control  kill 
Mission  kill  (MSN-kill) 

Mobility  kill  (M-kill) 


Personnel  kill 

Phase  kill 

Power  Supply  kill 

Prevent  Launch  kill  (PL-kill) 

Prevent  Mission  kill  (PM-kill) 

Prevent  Takeoff  kill  (PTO-kill) 

Short  Range  Sur.-to-Air  Firepower  kill 
Structural  kill 
Support  Functions  kill 
Surface-to-Air  Firepower  kill 
Thorough  Interdiction  kill 
Time  Out-of-Action  kill(TOA-kill) 


Many  of  these  kill  types  are  not  independent  or  exclusive  of  one  another  (i.e. 
a  target  may  have  more  than  one  kill  type  at  a  time).  Further,  some  kill  types 
do  not  apply  to  Army  (land)  engagements  or  cannot  be  ascertained  visually.  Most 
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importantly,  however,  CASTFOREM  only  models  a  subset  of  these  kill  types  and,  as 

such,  will  scope  the  consideration  of  the  research. 

CASTFOREM  models  the  following  kill  types: 

Mobility  kill  (M-kill)  A  target  is  subject  to  an  M-kill  if  it  is  incapable  of  executing 
controlled  movement  and  the  damage  is  not  repairable  by  the  crew  on  the  bat¬ 
tlefield.  Failure  to  function  may  be  caused  by  the  incapacitation  of  the  crew  or 
damage  to  propulsion  or  control  equipment. 

Firepower  kill  (F-kill)  A  target  is  subject  to  an  F-kill  if  it  is  incapable  of  delivering 
controlled  fire  from  the  main  armament  and  the  damage  is  not  repairable  by 
the  crew  on  the  battlefield.  The  loss  of  this  function  may  be  caused  by  the 
incapacitation  of  the  crew  or  damage  to  the  main  armament  and  its  associated 
equipment. 

Communications  kill  (C-kill)  A  target  is  subject  to  a  C-kill  if  it  is  incapable  of 
sustained  communications  with  other  battlefield  entities  and  the  damage  is  not 
repairable  by  the  crew  on  the  battlefield.  The  loss  of  this  function  may  be 
cause  by  crew  incapacitation  or  damage  to  communications  equipment.  A  C- 
kill  cannot  be  easily  perceived  visually  and  as  such  will  not  be  considered  in  this 
research. 

Sensor  kill  (S-kill)  A  target  is  subject  to  an  S-kill  if  it  is  incapable  of  using  its 
sensors,  offensive  or  defensive,  and  the  damage  is  not  repairable  by  the  crew  on 
the  battlefield.  The  loss  of  this  function  may  be  caused  by  crew  incapacitation 
or  damage  to  sensors  themselves.  An  S-kill  cannot  be  easily  perceived  visually 
and  as  such  will  not  be  considered  in  this  research. 

Catastrophic  kill  (K-kill)  A  target  is  subject  to  a  K-kill  if  it  sustains  both  an  M- 
and  F-kill  and  is  damaged  to  the  extent  that  is  not  economically  repairable.  A 
K-kill  is  more  likely  to  be  apparent  to  the  crew  of  a  weapon  system  because  of 
the  resulting  fires/detonation  of  ammunition. 
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No-kill  If  the  target  has  not  sustained  any  kill  type  and  is  capable  of  performing  all 

combat  functions  it  is  referred  to  as  No-kill.  This  is  most  often  referred  to  as 

No  Damage  (ND)  in  CASTFOREM. 

Any  combination  of  M-,  F-,  C-,  or  S-kills  are  possible  while  K-kill  and  No-kill 
are  singular  states.  For  example,  a  target  can  sustain  an  MF-kill  (i.e.  the  target  has 
mobility  and  firepower  kills  only),  but  cannot  have  a  MK-kill.  For  the  purposes  of 
this  research  kill  type  will  be  referred  to  as  kill  state. 

The  Army  Material  Systems  Analysis  Activity  (AMSAA)  provides  the  data 
required  to  derive  mutually  exclusive  probabilities  for  all  feasible  combinations  of  kill 
types  used  in  the  simulation.  It  is  important  to  note  that  CASTFOREM  does  not 
distinguish  between  reasons  for  a  kill.  For  example  a  vehicle  might  sustain  an  M-kill 
because  its  tracks  are  displaced,  the  engine  was  damaged,  or  the  soldier  driving  was 
killed,  but  CASTFOREM  only  knows  that  the  vehicle  has  an  M-kill. 

1.3  AMSAA  Heuristic 

CASTFOREM  currently  uses  a  heuristic  developed  by  AMSAA  to  model  BDA. 
The  heuristic  is  based  on  the  perception  of  the  soldier  in  combat  and  is  shown  in 
Figure  1.2  on  the  following  page. 

The  heuristic  begins  with  the  hrer  performing  the  first  shot  and  an  initial  eval¬ 
uation  of  the  target  state  as  K-killed  or  not  K-killed.  It  assumes  that  the  observer 
can  perceive  a  K-kill  with  probability  1.  As  a  result,  if  at  any  point  in  the  heuristic 
the  target  is  assessed  as  a  K-kill,  the  engagement  simulation  ends.  The  hrer  evalu¬ 
ates  non-K-kill  targets  again  with  a  0.33  probability  of  detecting  the  correct  level  of 
damage  and  0.67  probability  of  having  unknown  damage.  A  second  shot  is  performed 
and  assessed  in  the  same  manner  with  probability  of  correct  detection  increased  to 
0.67.  The  heuristic  ends  after  three  shots  where  non-K-killcd  targets  are  evaluated 
accurately  with  probability  1. 

The  AMSAA  heuristic  contains  several  inadequacies  that  need  to  be  addressed. 
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Figure  1.2:  Current  AMSAA  Heuristic 

1.  Assuming  that  a  K-kill  is  perceived  with  probability  1,  intuitively,  does  not 
make  sense.  Though  a  K-kill  is  the  easiest  kill  state  to  see  visually,  having  no 
error  in  this  determination  trivializes  the  difficulty  of  BDA  during  live  combat. 

2.  The  assignment  of  probabilities  to  detection  of  the  correct  kill  level  (less  than 
K-)  as  0.33,  0.67,  and  1  for  each  successive  shot,  respectively,  is  rather  arbitrary. 
Realistically,  there  is  no  guarantee  of  detecting  the  correct  kill  state  after  some 
number  of  shots. 

3.  The  algorithm  does  not  take  the  firer’s  perception  of  the  targets  actions  into 
account.  Situational  factors,  such  as  the  target’s  movement  or  if  it  is  engaging 
friendly  forces,  will  obviously  affect  the  assignment  of  a  kill  state  to  an  enemy 
target. 

4.  No  probabilities  are  associated  with  assessing  the  target  to  specific  kill  types 
(e.g.  M,  F,  MF,  K,  etc.).  Further,  an  assessment  as  unknown  results  in  the 
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same  action  (another  shot)  as  assessing  the  target  as  less-than-K.  Consequently 
having  an  unknown  assessment  may  be  unnecessary. 

5.  The  heuristic  lacks  any  representation  of  indirect  fire,  including  mortar,  missile, 
or  air  attacks. 

6.  Environmental  factors  such  as  weather,  time  of  day,  range,  sensor,  terrain,  and 
obscurants  are  not  taken  into  account.  Clearly,  these  factors  would  affect  or 
degrade  the  hrers  ability  to  make  an  accurate  assessment. 

1.4  Research  Objectives 

The  objectives  of  this  research  are  as  follows: 

1.  Develop  a  methodology  to  model  the  BDA  process  from  a  perception  standpoint 
that  addresses  the  inadequacies  of  the  current  heuristic. 

2.  Develop  an  efficient  SMEs  survey  instrument  that  will  produce  the  form  of  data 
needed  for  the  proposed  methodology  and  give  TRAC  an  initial  situational  BDA 
data  set. 

3.  Propose  a  technique  to  implement  the  methodology  into  a  combat  model. 

1.5  Method  of  Approach 

For  the  purposes  of  CASTFOREM,  this  research  focuses  on  the  collection  and 
processing  of  BDA  information  during  an  ongoing  engagement.  The  typical  situation 
involves  a  dismounted  soldier  or  gunner  and  his  decision  to  fire  again  based  on  sensory 
and  environmental  information.  The  main  goal  is  to  develop  a  mathematical  model 
which  can  be  implemented  in  CASTFOREM  that  accurately  represents  the  real  time 
BDA  process.  It  will  focus  on  the  hrer’s  perception  of  the  battlefield  scenario  and 
his  assignment  of  a  kill  state  based  on  the  information  available.  To  accomplish  this, 
a  set  of  event  trees  will  be  built  to  characterize  the  possible  situations  that  could 
occur  on  the  battlefield.  Situations  (i.e.  end  nodes  of  the  trees)  will  then  form  a  set  of 
parameters  that  will  frame  questionnaires  for  subject  matter  experts.  The  responses 


7 


will  yield  an  initial  data  set  subsequently  used  to  develop  probability  distributions  for 
kill  types  conditioned  on  the  situational  parameters. 

Since  the  model  will  be  from  the  hrer’s  perspective,  the  kill  types  that  CAST- 
FOREM  models  must  be  reduced  to  those  that  can  be  perceived  visually.  C-kill  and 
S-kill  will  not  enter  consideration  in  the  model  because  no  tractable  method  exists  to 
visually  asses  these  kill  types  on  the  battlefield.  This  leaves  four  basic  kill  types  (M- 
kill,  F-kill,  K-kill,  and  ND)  and  one  kill  combination  (MF-kill).  The  model  will  assign 
probabilities  of  assessing  a  target  as  each  of  these  kill  types,  based  on  the  scenario 
facing  the  hrer. 

1 . 6  Organization 

This  thesis  is  divided  into  five  chapters.  Chapter  1  presented  the  research 
topic,  the  background  behind  it,  and  its  motivation.  Further,  CASTFOREMs  current 
method  for  modeling  BDA  and  its  limitations  were  discussed  and  a  brief  overview  of 
the  new  approach  was  discussed. 

Chapter  If  will  review  the  relevant  literature  on  BDA  and  combat  modeling  as 
well  as  present  an  overview  of  the  CASTFOREM  engagement  process.  Chapter  Iff 
details  the  methodology  used  for  data  collection,  develops  the  mathematical  foun¬ 
dations  for  the  proposed  BDA  model,  and  suggests  an  strategy  for  implementation 
into  a  combat-simulation  model.  Chapter  IV  provides  a  statistical  evaluation  of  the 
methodology  using  survey  data.  Lastly,  Chapter  V  presents  the  conclusions  and  rec¬ 
ommendations  of  this  work  and  suggests  future  research  in  this  area. 


II.  Literature  Review 


2. 1  Introduction 

Modeling  BDA  explicitly  in  combat  models  is  a  relatively  young  concept.  By  ex¬ 
cluding  the  BDA  process  from  combat  simulations,  enemy  battle  damage  was  treated 
as  a  known  quantity.  Of  course,  the  BDA  process  contains  a  great  deal  of  uncertainty 
and  can  heavily  influence  the  flow  of  an  engagement.  The  effects  of  the  BDA  process 
on  a  military  conflict  are  significant,  and  in  the  arena  of  close  combat  the  process  is 
both  gravely  important  and  largely  uncertain.  In  the  heat  of  battle,  BDA  is  dynamic 
and  the  wrong  determination  may  lead  to  a  unit’s  demise. 

While  little  has  been  done  to  explain  the  BDA  process  in  general,  even  less  exists 
to  describe  the  intricacies  of  close  combat.  Most  of  the  literature  addresses  the  effects 
of  timeliness  and  accuracy  of  BDA  on  an  air  campaign  and  its  respective  targeting 
process.  Though  the  BDA  process  for  air  operations  is  widely  different  in  time  frame 
(e.g.  assessments  are  generally  made  after  a  mission)  it  shares  a  similar  uncertain 
nature  with  close  combat  situations.  This  chapter  surveys  the  relevant  BDA  issues 
(e.g.  information  gain,  uncertainty)  as  well  as  the  methodologies  proposed  to  model 
it  in  combat  simulation  models.  Further  the  CASTFOREM  methodologies  related  to 
engagement  and  BDA  are  reviewed. 

2. 2  BDA  Information 

The  value  of  BDA  information  in  combat  situations  has  expanded  greatly  since 
Operation  Desert  Storm  in  the  early  1990’s.  The  increased  tempo  of  operations  in 
a  new  era  of  warfare  along  with  the  increasing  use  of  precision  guided  munitions 
(PGMs)  make  BDA  information  critical  to  the  efficient  application  of  forces.  Yost 
and  Washburn  [21]  expound  on  this  concept  in  relation  to  allocating  assets  on  the 
modern  battlefield. 

Baird  et  al.  [1]  researched  the  value  of  information  gain  on  the  battlefield.  Their 
work  included  a  study  of  how  accuracy,  timeliness,  and  completeness  of  information, 
including  BDA,  target  type,  and  target  location,  affected  the  number  of  munitions 
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expended  in  a  simulation.  A  regression  model  approach  showed  intuitive  results;  the 
largest  gain  in  effectiveness  was  obtained  from  the  interaction  of  high  accuracy  and 
more  timely  data. 

Manor  and  Kress  [15]  present  an  algorithm  for  engaging  targets  with  incomplete 
BDA  information.  The  results  show  that  engaging  the  targets  using  a  greedy  strategy 
(i.e.  shooting  at  the  least  prior  engaged  target)  maximizes  the  effectiveness  per  round. 
Though  the  greedy  approach  maximizes  this  objective,  success  in  actual  combat  does 
not  depend  on  effectiveness  per  round.  This  differs  from  the  current  Army  modeling 
technique  by  re-targeting  after  each  shot  and  reflects  the  differing  goals  of  the  models. 

Song  [17]  also  investigated  the  value  of  BDA  information  to  make  ballistic  missile 
defense  systems  more  effective.  Though  the  model  dealt  with  missile  defense,  it 
stressed  the  importance  of  BDA  information  for  decision  making  in  a  time-critical 
environment,  a  similar  situation  to  close  combat.  Additionally,  Song  notes  the  role  of 
information  in  dealing  with  the  uncertainty  of  a  cognisant  enemy. 

2.3  Information  and  Uncertainty 

The  method  proposed  in  this  thesis  models  BDA  from  a  perceptional  standpoint 
in  order  to  capture  uncertainty  in  the  process.  The  uncertainty  involved  in  the  BDA 
process  has  been  stated  extensively  in  outlining  the  shortcomings  of  conflicts  in  recent 
years  [18].  This  uncertainty  stems  not  only  from  the  adversary,  but  also  from  friendly 
force  miscues.  The  amount  of  communications  and  integration  required  among  coali¬ 
tion  forces  to  execute  effective  BDA  makes  the  process  extremely  complex  and  prone 
to  error,  leading  to  missed  (or  assumed  dead)  targets  and  over-killed  targets  [18]. 

Modeling  the  BDA  process  is  essentially  an  exercise  in  modeling  perceived  infor¬ 
mation  and  uncertainty.  As  such,  the  method  proposed  for  this  research  relies  on  the 
link  between  information  theory  and  the  human  thought  process.  Jaynes  [9]  connects 
human  reasoning  to  Laplace’s  model  of  common  sense,  and  in  turn,  to  the  concepts 
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of  maximum  entropy  and  Bayesian  inference.  Jaynes  [7]  [14]  [8]  has  also  completed 
extensive  work  on  maximum  entropy  as  well  as  its  connections  to  Bayes  theorem  [10]. 

Jelinck  [11]  used  Markov  chains  to  model  uncertainty  in  combat  operations. 
BDA  is  a  function  of  perceived  information  and  its  uncertainty.  Gaver  et  al.  [6]  dealt 
with  the  incorporation  of  uncertainty  in  perception  information  into  combat  models. 
Combat  perceptions  were  modeled  stochastically  to  capture  the  inherent  uncertainty 
in  combat  information. 

2.4  BDA  Modeling  Concepts 

Several  tchniques  for  modeling  BDA  have  been  developed  in  recent  years,  though 
none  specifically  for  combat  simulation  models.  Franzen  [2]  created  a  BDA  model 
for  air  campaign  targeteers  based  on  a  Bayesian  belief  network.  The  Bayseian  net 
incorporated  the  addition  of  information  into  the  BDA  process  after  a  target  is  struck. 
Subject  matter  experts  (SMEs)  estimated  initial  conditional  probabilities,  which  were 
updated  using  the  data  learning  property  of  Bayesian  nets.  Though  this  method  is 
centered  around  air  operations,  it  offers  some  parallels  to  the  proposed  method  in 
using  Bayesian  techniques  to  determine  probabilities  of  damage. 

Gaver  and  Jacobs  [4]  developed  a  Shoot-Look- Shoot  approach  to  engaging  targets 
and  a  simple  formulation  of  the  BDA  problem.  The  formulation  treated  targets 
simply  as  alive  or  dead  and  gave  conditional  probability  distributions  of  perceiving 
the  correct  state  given  the  ground  truth.  Tradeoffs  between  probability  of  kill  (pk) 
and  BDA  information  accuracy  were  investigated  for  several  tactics  including  Shoot- 
Look-Shoot  and  deterministically  shooting  two  shots.  In  a  piggyback  effort,  Gaver  and 
Jacobs  [5]  showed  the  effects  of  BDA  accuracy  in  the  Shoot-Look-Shoot  construct  on 
a  service  queue  of  possible  targets.  This  iterative  process  of  engaging  a  target  closely 
resembles  the  algorithm  used  in  current  Army  combat  models.  Though  the  BDA 
problem  formulation  is  simplistic  (i.e.  targets  are  either  alive  or  dead),  it  stands  as  a 
good  example  of  a  probabilistic  modeling  technique  for  the  BDA  process. 
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The  BDA  modeling  technique  proposed  in  this  research  uses  Markov  chains  to 
capture  uncertainty  in  what  might  happen  after  firing  at  a  target  and  Bayesian  infer¬ 
ence  to  link  perceived  information  about  the  target  to  some  probability  distribution  of 
kill  states.  This  method  will  be  implemented  within  the  construct  of  CASTFOREM 
and  as  such  warrants  a  brief  overview  of  the  pertinent  modules  within  the  model. 

2.5  CASTFOREM 

CASTFOREM  is  the  Army’s  current  force-on- force  model  intended  to  simulate 
conflicts  of  60  minutes  or  less.  Conflicts  take  place  between  two  main  forces,  Blue 
(friendly),  and  Red  (enemy)  respectively.  It  is  an  agent-based  model  comprised  of 
organizational  entities  independently  interacting  according  to  their  logic  set  (i.e.  in¬ 
telligence),  comprised  of  orders  and  decision  tables.  The  model  executes  interactions 
at  the  weapon  system  level  and  can  simulate  complex  systems  such  as  communi¬ 
cations  and  logistics  networks.  Furthermore,  CASTFOREM  can  operate  at  several 
levels  of  fidelity  depending  on  the  size  of  individual  unit  modeled  (e.g.  single  soldier 
vs.  weapon  system  with  a  crew). 

During  a  conflict,  entities  within  the  simulation  interact  in  many  different  ways 
according  to  their  logic  set.  The  BDA  process  in  CASTFOREM  is  interdependent 
with  most  of  these  modules,  but  relies  heavily  upon  only  a  select  few:  target  search  and 
acquisition,  probability  of  hit  (PH)  calculations,  probability  of  kill  (PK)  calculations, 
and  the  response  algorithms  for  both  firer  and  target  (i.e.  the  engagement). 

2.5.1  Target  Search  and  Acquisition.  In  order  for  a  blue  entity,  such  as  an 
armored  personnel  carrier  (APC),  to  engage  a  target,  it  must  first  create  a  list  of 
targets  by  searching  the  battle  space.  A  general  flow  chart  of  the  search  process  is 
given  in  Figure  2.1  on  the  next  page. 

The  simulation  provides  target  detection  and  combat  identification  (CID)  us¬ 
ing  direct  view  optics  (DVO),  image  intensifiers,  television  (TV),  and  infrared  (IR) 
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of  Search  Logic  [19] 


sensors.  CASTFOREM  handles  this  search  process  and  CID  though  the  ACQUIRE 
model. 

The  ACQUIRE  model  simulates  the  physics  involved  in  entities  viewing  a  tar¬ 
get.  It  uses  several  mathematical  models  to  determine  how  visible  a  target  is  to  an 
observer,  taking  into  account  current  atmospheric,  environmental,  and  physical  con¬ 
ditions.  Atmospheric  conditions  might  include  weather  (e.g.  day/night,  cloud  cover, 
humidity)  or  dust  and  smoke.  These  conditions  are  accounted  for  in  the  Combined 
Obscuration  Model  for  Battlefield  Induced  Contaminates  (COMBIC).  The  environ¬ 
mental  conditions  represent  such  things  as  line  of  sight  (LOS),  range  to  the  target, 
and  the  nature  of  the  tactical  area  (forest,  desert,  urban,  etc).  Lastly,  the  observer’s 
field  of  regard,  sensor  type,  target  type,  and  movement  (both  observer  and  target) 
characterize  physical  conditions.  ACQILIRE  combines  the  effects  of  atmospheric,  en¬ 
vironmental,  and  physical  conditions  on  CID  of  a  target  and  makes  a  determination. 

The  ACQLIIRE  model  provides  an  output  at  one  of  four  levels  for  target  acqui¬ 
sition.  The  levels  are  defined  in  [19]  as: 

Detection  A  target  of  military  interest  has  been  acquired. 

Classification  A  type  of  target  (e.g.  wheeled,  tracked,  stationary)  has  been  detected 
and  an  aimpoint  can  be  determined  on  the  detected  target. 

Recognition  The  target  class  (e.g.  tank  versus  APC)  can  be  resolved  on  the  detected 
target. 

ID  Call  The  observer  thinks  he  has  sufficient  resolution  on  the  detected  target  to 
make  an  identification,  which  may  or  may  not  be  correct  (e.g.  T-72  tank,  M2A3 
Bradley) . 

The  entity  keeps  a  list  of  target  candidates  and  their  respective  acquisition  levels 
within  its  field  of  regard  (FOR)  at  any  one  time.  This  list  is  rank  ordered,  based  on 
several  parameters  including  proximity,  target  contrast,  and  threat.  An  entity  will 
choose  the  most  logical  target  for  engagement  and  proceed  according  to  its  logic  set. 
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It  is  important  to  note  that  an  engagement  may  occur  at  any  of  the  acquisition  levels 
due  to  knowledge  of  which  force  (Blue  or  Red)  controls  the  area  where  the  target  was 
detected. 

2.5.2  PH/PK  Calculation.  When  shots  are  exchanged  between  entities, 
CASTFOREM  must  first  determine  if  the  target  was  hit  (i.e.  PH).  If  the  round 
impacts  the  target  the  model  must  subsequently  determine  the  appropriate  level  of 
damage  (i.e.  PK).  CASTFOREM  handles  PH  and  PK  in  different  ways  depending 
on  the  type  of  entity  involved.  A  vehicle  and  its  crew  are  assessed  as  one  in  that  kills 
may  result  from  either  damage  to  the  vehicle  or  an  injury  to  its  operator.  Mounted 
personnel  (i.e.  on  a  vehicle)  share  PH  calculations  with  their  vehicle  but  assess  PK 
independently.  Based  on  the  level  of  detail  in  a  particular  model  run,  dismounted 
personnel  may  be  treated  as  aggregated  or  individually. 

CASTFOREM  represents  a  vehicle  as  two  cell  approximations  of  its  turret  and 
hull  as  depicted  in  Figure  2.2.  Si  through  S4  are  examples  of  impact  points  determined 


Figure  2.2:  Two  Cell  Approximation  of  CASTFOREM  Unit  [19] 

using  the  vehicle  silhouette,  an  aimpoint  bias,  and  round  dispersion.  Impact  points 
Si  and  S4  are  misses,  S2  is  a  hull  hit,  and  S3  is  a  turret  hit. 
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A  vehicle’s  silhouette  is  simply  the  two  dimensional  projection  of  the  vehicle 
from  a  side  view,  taking  cover  into  account.  Generally,  the  center  of  mass  for  the 
silhouette  is  the  unbiased  aimpoint  for  a  particular  shot.  CASTFOREM  biases  the 
aimpoint  for  the  round  and  then  models  round  dispersion  as  a  bivariate  normal  distri¬ 
bution,  centered  on  the  new  biased  point.  AMSAA  provides  data  for  weapon  system 
aimpoint  biases  and  munition  specific  dispersion  information.  Figure  2.3  displays  an 
example  of  this  system.  So,  each  round  is  a  draw  from  its  specific  bivariate  normal 


distribution  and  hits  are  recorded  for  those  draws  that  land  on  the  target  silhouette. 

After  determining  that  a  particular  round  hits  its  target,  damage  must  be  as¬ 
signed  to  the  entity  through  PK  calculations.  The  impact  assessment  assigns  damage 
to  several  target  systems  and  evaluates  the  effects  on  the  targets’  combat  capabilities. 
Targets  that  lose  one  or  more  capabilities  due  to  damage  will  have  kill  types  asso¬ 
ciated  with  them.  Damage  here  should  be  thought  of  as  Ground  Truth  rather  than 
BDA  perceptions. 

PK  determinations  are  handled  in  two  different  ways  for  dismounted  personnel. 
Determining  PK  data  for  aggregated  dismounted  personnel  is  a  function  of  the  size 
of  the  unit,  round  type,  range  from  the  impact,  and  their  tactical  posture  at  the 
time  of  the  round  impact.  CASTFOREM  can  also  perform  attrition  of  personnel  as 
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individual  entities.  For  this  resolution,  dismounted  soldiers  are  physically  represented 
as  sets  of  cylinders,  depicted  in  Figure  2.4  and  Figure  2.5  on  the  next  page. 


Figure  2.4:  Dismounted  Soldier  Representation  (Side  View)  [19] 


If  a  soldier  is  hit,  the  damage  depends  on  exactly  where  and  how  the  round 
passes  through  the  body.  For  example,  a  shot  to  the  trunk  may  cause  a  K-kill  while 
a  shot  in  the  legs  might  result  in  a  M-kill.  How  much  of  the  body  is  hit  (e.g.  center 
of  mass  vs.  graze)  also  affects  how  damage  is  assigned.  CASTFOREM  uses  an  inca¬ 
pacitation  kill  to  address  the  situation  where  soldiers  can  perform  for  a  limited  time 
after  receiving  wounds.  Soldiers  are  allowed  combat  functionality  for  one  of  four  time 
periods  before  being  removed  from  the  simulation:  5  seconds,  30  seconds,  5  minutes, 
or  30  minutes  (depending  on  the  severity  of  the  wounds).  In  essence,  the  less  body 
mass  a  round  passes  through,  the  more  incapacitation  time  is  given  to  the  soldier. 

2.5.3  Firer /Target  Response  Algorithms.  An  engagement  in  CASTFOREM 
consists  of  one  entity  bring  a  sequence  of  rounds  at  an  enemy  entity.  Within  each 


17 


X-Y 


extremities 
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Figure  2.5:  Dismounted  Soldier  Representation  (Top  View)  [19] 

engagement,  the  firer  and  target  have  response  algorithms  to  complete  the  interactions 
after  PH  and  PK  calculations  are  complete. 

The  target,  upon  receiving  impact  damage  as  described  above,  is  given  the 
chance  to  respond  to  taking  fire,  unless  the  target  sustained  a  K-kill.  The  target 
uses  its  response-to-fire  decision  tables  to  simulate  a  response  which  may  include 
suppressed  combat  effectiveness.  Additionally,  other  actions  may  be  taken,  such  as 
moving,  covering,  retreating,  or  a  number  of  other  actions. 

The  firer,  after  impact  of  its  round,  first  attempts  to  perceive  the  level  of  damage 
inflicted  on  the  target.  This  is  where  the  BDA  algorithm  is  performed.  If  the  desired 
level  of  damage  is  perceived,  the  firer  uses  its  end-of-engagement  decision  tables  to 
determine  the  entity’s  next  action.  The  next  action  might  consist  of  engaging  another 
target  within  the  FOR,  moving  to  a  collection  point,  or  communicating  with  command 
entities.  Figure  2.6  on  the  following  page  shows  an  example  flow  chart  of  the  end-of- 
engagement  logic  flow. 


18 


<? 


Figure  2.6:  Tank  Engage  Complete  Logic  Flow  [19] 
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If  the  desired  level  of  damage  is  not  perceived,  the  hrer  may  take  another  shot 
to  achieve  its  goal,  if  it  is  allowed.  CASTFOREM  allows  only  np  to  three  shots  in 
succession  in  any  one  engagement  due  to  a  modeling  decision.  The  model  wishes 
to  prevent  bring  too  many  rounds  at  one  single  target,  and  therefore  expending  the 
available  ammunition  on  a  small  set  of  targets. 

2. 6  conclusion 

In  this  chapter,  a  brief  overview  of  the  concepts  related  to  the  BDA  process 
was  provided.  This  research  will  be  the  most  detailed  representation  of  real  time 
BDA  in  close  combat  to  date.  As  a  result,  the  literature  is  not  well  established  in 
this  area.  First,  a  review  of  BDA  as  an  information  currency  was  explored.  Then, 
precedents  for  using  information  theory  and  modeling  uncertainty  as  a  stochastic  pro¬ 
cess  were  presented.  Additionally,  several  former  techniques  for  modeling  BDA  were 
stated.  Lastly,  the  relevant  CASTFOREM  engagement  methodologies  were  described 
in  detail. 
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III.  Methodology 


3. 1  Introduction 

The  BDA  process  has  found  much  difficulty  in  accurately  translating  its  com¬ 
plexities  into  a  model.  As  a  result,  the  process  has  been  modeled  poorly  or  completely 
left  out  of  most  combat  models.  In  the  following  sections,  a  methodology  to  effectively 
model  the  BDA  process  will  be  mapped  out  from  data  collection  to  implementation. 

This  chapter  first  presents  a  method  to  collect  the  appropriate  data.  Second, 
a  methodology  for  modeling  the  BDA  process  as  a  Markov  Process  is  presented. 
Then,  a  method  for  updating  the  process  based  on  observed  conditions  using  Bayesian 
inference  is  shown.  Subsequently,  the  application  and  integration  of  the  methodology 
to  the  combat  model  are  proposed.  Finally,  example  calculations  are  completed  to 
illustrate  the  method. 

3.2  Data  Collection 

Since  no  precedent  exists  for  real  time  BDA  in  combat  simulation  models,  devel¬ 
opment  of  an  adequate  initial  data  set  is  needed.  To  develop  an  accurate  representa¬ 
tion  of  the  BDA  process,  data  needs  to  be  from  the  hrer’s,  (i.e.  soldier’s)  perception. 
Since  the  scope  of  this  project  is  mostly  vehicles  engaging  other  vehicles  (i.e.  tanks, 
APCs,  trucks,  etc.),  a  set  of  event  trees  was  developed  to  fully  represent  the  possible 
actions  and  physical  damage.  These  trees  served  to  frame  a  set  of  survey  questions 
for  subject  matter  experts  (SMEs). 

3.2.1  Survey  Participants.  To  develop  the  appropriate  data,  it  is  important 
to  survey  veterans  of  live  combat  as  SMEs.  The  SMEs  should  include  a  collection  of 
Army  combat  veterans  from  various  career  fields.  The  SMEs  combat  experience  is 
extremely  important  to  understanding  the  thought  process  of  a  shooter  (i.e.  someone 
on  the  trigger)  while  engaging  a  target,  performing  BDA,  and  subsequently  deciding 
to  fire  again.  Further,  their  experience  in  viewing  targets  having  various  levels  of 
damage  will  be  integral  to  collecting  the  pertinent  data. 
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3.2.2  Survey  Design.  The  SMEs  will  provide  qualitative  predictions  for 
actions  and  visible  damage  after  targets  take  fire.  Before  detailing  the  proposed 
survey  questions  and  perspective  responses,  several  terms  referred  to  in  this  section 
must  be  defined. 

Event  More  specifically  called  an  Elementary  Event,  it  is  the  particular  action  or 
level  the  target  being  assessed  presents;  essentially  analogous  to  an  outcome  of 
a  chance  event. 

Event  Set  A  set  of  mutually  exclusive  and  collectively  exhaustive  events  a  target 
can  assume  (e.g.  Engaging,  Not  Engaging). 

Situation  The  particular  combination  or  intersection  of  events  that  a  target  possesses 
at  the  time  of  an  assessment. 

Kill  State  Referred  to  by  DoD  as  Kill  Type,  it  is  the  type  of  kill  (ND,  M,  F,  MF, 
or  K)  assigned  to  the  target  during  an  assessment.  The  kill  types  are  both 
mutually  exclusive  and  collectively  exhaustive. 

It  is  important  to  note,  that  the  form  of  data  collected  in  survey  responses 
reflects  the  modeling  technique  presented  in  this  research.  Different  methodologies 
would  require  different  survey  structures.  In  that  light,  this  survey  was  designed 
specifically  to  obtain  the  data  relevant  to  modeling  the  BDA  process  as  a  Markov 
chain  with  Bayesian  updates. 

Survey  participants  will  be  asked  to  assign  qualitative  values  to  several  events 
given  a  situation  which  included  the  target  type  and  its  respective  kill  state.  Fig¬ 
ure  3.1  on  the  next  page  shows  an  example  of  a  response  table  (for  a  tank)  that  the 
SME  would  complete. 

Movement,  Engaging  Targets,  Activity,  Turret,  Hull,  and  Tracks  are  all  event 
sets.  Movement,  Engaging  Targets,  and  Activity  refer  to  observations  of  the  target’s 
tactical  responses.  The  Turret,  Hull,  and  Tracks  event  sets  measure  the  (visible) 
physical  damage  on  each  component  of  the  tank.  The  event  sets  respective  events 
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Actions 

•  Movement 

o  Toward  your  position 
o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engagement 

o  Engaging  Targets 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Tank 
o  Other/No  Action 


N  M  A  Physical  Damage 

•  Turret 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

~  1j  Zl  •  Hul1 

o  No  Damage 

□  □□  o  Light  Damage 

o  Heavy  Damage 

•  Tracks 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


N  M  A 


Figure  3.1:  Sample  Response  Table 


(e.g.  No  Damage,  Light  Damage,  or  Heavy  Damage  for  Turret)  are  mutually  exclusive 
and  collectively  exhaustive,  meaning  only  one  of  the  events  in  each  set  may  happen 
to  the  target  at  any  time. 

Within  each  of  the  event  sets,  the  SME  is  required  to  mark  the  events  with  one 
of  three  qualitative  assessments:  events  the  soldier  would  never  see  (N  column),  ones 
he  might  see  (M  column),  or  those  he  would  always  see  (A  column). 

Response  tables  differed  according  to  the  enemy  vehicle  type  tailored  to  the 
appropriate  vehicle  components.  The  survey  also  provided  schematic  diagrams  of  the 
enemy  vehicles  for  soldiers  to  illustrate  specific  points  of  interest  and  space  to  add 
important  events  or  event  sets  that  were  excluded  from  the  response  tables.  A  copy 
of  the  questionnaire  is  provided  for  the  reader  in  appendix  A. 


3.3  Modeling  the  BDA  Process  as  a  Markov  Chain 

A  system  changing  randomly  over  time  can  be  represented  as  a  sequence  of 
random  variables,  X  =  { An,  n  >  0},  where  Xn  denotes  the  state  of  the  system  at 
time  n.  If,  for  all  n,  Xn  must  exist  in  the  finite  set,  S  =  {1,2, . . . ,  m},  {Xn,n  >  0} 
is  called  a  time  series  stochastic  process  with  state  space  S.  This  process  can  further 
be  modeled  as  a  Discrete  Time  Markov  Chain  (DTMC)  if  it  satisfies  the  following 
conditions. 
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1.  For  all  n  >  0,  Xn  =  in  e  S  with  probability  1.  That  is,  the  system  must  exist 
in  one  of  its  states  (kill  types)  at  all  times. 

2.  For  all  n  ^  0,  Xn+i  Xn  in,  Xn—\  in_i,  ■  •  -%'Xq  io  G  S , 


P{An_|_i  .)  in,  in_i,  .  .  .  ,  Ag  *0}  -^{-An+1  j\Xn  in\ 


That  is,  the  the  probability  of  a  state  at  time  (n  +  1)  depends  only  on  the  nth 
state.  This  is  known  as  the  Markovian  property. 

Further,  the  DTMC  {Xn,  n  >  0}  is  time  homogeneous  if  the  conditional  probabilities, 
P{Xn+i  =  j  | Xn  =  i}  do  not  depend  on  n.  This  time  homogeneity  can  be  expressed 
mathematically  as 

Pij(n)  =  pij  V  n  >  0  i,j  e  S 

where  represents  P{Xn+ 1  =  j\Xn  =  i}.  Now,  let  P  =  \j)t]\  denote  the  one 
step  transition  probability  matrix  for  the  DTMC  {Xn,n  >  0}.  For  the  case  where 
S  =  {1,2,...,  m}.  P  can  be  represented  as 


P  n  p  12 


Pl,m—  1  Plm 


P  = 


P  21  P22 


P2,m~l  P2m 


Pm— 1,1  Pm— 1,2 


Pm—  l,m— 1  Pm—l,m 


Pml  Pm2 


Pm. w  1  Pmm 


for  which 

Pij  =  1  v  *  e  s. 

ies 

This  means  that,  since  |Xn,  n  >  0}  is  a  DTMC,  P  is,  by  definition,  stochastic. 
Modeling  the  BDA  process  as  a  DTMC  gives  the  advantage  of  using  transient  analysis 
to  determine  shot  dependent  probability  distributions  for  each  assessment. 
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3.3. 1  Transient  Analysis  of  a  DTMC.  In  order  to  perform  transient  analysis, 
the  DTMC  must  be  fully  characterized  with  P  and  an  initial  distribution  for  Xq.  Let 
a^  be  defined  as 

a(0)  =  (a-0))  V  i  G  S. 

That  is,  a^°)  represents  a  row  vector  of  the  initial  distribution  (pmf)  of  Xq.  Now,  with 
P  and  a(0\  the  distribution  of  Xn  at  any  time  n  can  be  calculated.  First,  let 

=p{Xn=]}  =  YJP{Xn=j\XQ  =  i}P{XQ=i}  V  jeS  (3.1) 

ieS 

be  the  probability  of  {Xn,  n  >  0}  existing  in  state  j  at  time  n  which  can  be  represented 
in  vector  form  by  a^d.  Further,  let 

Pij]  =  pixn  =j\x0  =  i}  i,j  e  S 

denote  the  n  step  transition  probability  from  i  to  j.  Recall  that  P  is  the  one  step 
transition  probability  matrix.  If  P2  represents  the  two  step  transition  matrix,  it 
follows  that  the  case  for  some  n  steps  is 

P(n)  =  Ip!”’] 

_  pTl 


So  it  follows  from  Equation  3.1  that 

a(n)  =  a(0)P(n).  (3.2) 

The  transient  analysis  is  not  limited  to  conditioning  on  the  initial  distribution. 
The  distribution  a^"-*  is  easily  calculated  from  the  distribution  at  any  intermediate 
time  k.  The  result  is  an  analogue  of  Equation  (3.2). 

a(n)  =  a(fc)P(n-fc).  (3.3) 
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For  a  more  rigorous  demonstration  of  transient  analysis  of  a  DTMC,  the  reader  is 
directed  to  [13]. 


3.3.2  Representation  of  The  BDA  Process.  Now,  this  research  aims  to  model 
the  BDA  process  as  a  DTMC,  {Xn,  n  >  0}  where  Xn  denotes  the  assessment  after 
the  nth  shot,  with  state  space  S  =  {  ND,M,F,MF,K}  (i.e.  the  set  of  kill  states).  To 
determine  P  for  this  process,  consider  that  only  certain  state  transitions  (is  to  j s)  are 
feasible  in  reality. 

From  a  ground  truth  standpoint,  assume  that  within  a  single  engagement  (made 
up  of  one  or  more  consecutive  shots),  the  target  cannot  regress  in  damage  due  to 
actions  such  as  repairs.  This  is  a  reasonable  assumption  because  major  system  repairs 
will  not  occur  (or  at  least  are  highly  unlikely)  in  the  heat  of  a  battle.  This  assumption 
means  that  once  a  target  sustains  one  type  of  kill,  the  target  must  have  at  least  that 
kill  state  in  the  next  shot  iteration.  To  illustrate,  let  Xn  be  the  true  state  of  the 
target  after  nth  shot.  A  target  cannot  transition  from  Xi  =  M  kill  state  to  X2  =  F 
kill  state,  because  Xn  =  F-kill  implies  that  the  target  is  F-killcd  only,  and  thus  could 
not  have  transitioned  from  M-kill  at  n  —  1.  If  the  target  sustained  F-kill  damage  at 
n  —  2  it  would  instead  transition  to  X2  =MF  kill  state  to  include  the  M-kill  from  X\. 
As  a  result,  a  target  may  only  transition  to  a  subset  of  the  possible  kill  states  on  a 
second  shot  from  any  given  kill  state  on  a  first  shot.  For  instance,  if  a  target  sustains 
an  F-kill  on  the  first  shot  (n  =  1)  it  may  only  transition  into  F-,  MF-,  or  K-kill  states 
after  a  second  shot  (n  =  2).  Table  3.1  shows  the  possible  transitions  from  each  kill 
state. 


Table  3.1:  Possible  Kill  State  Transitions 


Kill  State  (Xn  =  i ) 

Feasible  Kill  State  Transitions  {Xn+\  =  j ) 

ND 

ND,  M,  F,  MF,  K 

M 

M,  MF,  K 

F 

F,  MF,  K 

MF 

MF,  K 

K 

K 
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With  no  knowledge  of  what  may  result  from  an  additional  shot  on  a  target, 
assigning  equal  probabilities  to  each  feasible  transition  is  justified  by  the  principle  of 
maximum  entropy  within  information  theory. 

Information  entropy  was  developed  by  Shannon,  who  used  it  as  a  way  to  measure 
the  amount  of  uncertainty  in  the  outcome  of  a  chance  event.  The  entropy,  denoted 
by  H,  of  a  random  variable  X  is  a  function  of  the  set  of  probabilities,  pi,P2,  ■  ■  ■  ,Pr, 
corresponding  to  the  r  possible  states  X  can  take  on.  H  is  stated  mathematically 
in  [16]  as 

r 

H  =  -K  pi  log  Pi 
1=1 

where  K  is  a  positive  constant  and  log(*)  is  any  logarithmic  function.  H  can  take 
on  many  forms  depending  on  the  definition  of  K  to  scale  entropy  and  give  it  a  unit 
of  measure.  Thus,  entropy  can  be  stated  without  dimensions  as  H  =  —  ^p<:logpj. 
One  significant  property  of  Shannon’s  entropy  is  that  entropy  is  maximized  when  a 
probability  distribution  is  uniform. 

Information  entropy  parallels  thermodynamic  entropy,  where  it  denotes  the 
amount  of  randomness  in  a  system.  Jaynes  [7]  proposed  that  thermodynamics  demon¬ 
strated  only  an  instance  of  information  theory  and  entropy.  Thus,  the  principle  of 
maximum  entropy  was  extended  from  thermodynamics  as  a  mathematical  basis  for 
Laplace’s  principle  of  insufficient  reason.  The  principle  of  maximum  entropy  states 
that  the  least  biased  model  is  the  one  that  maximizes  entropy  (uncertainty)  while 
remaining  consistent  with  the  prior  information. 

Now,  consider  maximizing  entropy  for  the  discrete  probability  distribution  above 
(p\,p-2,  ■  ■  ■  ,pr )•  Mathematically  this  problem  is  stated  as 

r 

max  /  {jpi ,  p2 ,  •  •  •  ,pr)  =  ~  ^  Pk  In  pk 

k=  1 
r 

s.t.  y,  Pk  =  1.  (3.4) 

k= 1 
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This  maximization  can  be  solved  by  the  method  of  Lagrange  multipliers. 


~ —  (/  +  A 9 )  —  0 
OPk 


d 

dpk 


-J^pfclnpfc  +  A  E  Pk  -  1 


fc=i 


,fe=i 


=  0 


-(in  pk  +  1)  +  A  =  0 


Pk  =  eA+1 


(3.5) 


Since  Equation  (3.5)  depends  only  on  A,  each  pk  is  equal.  Consequently,  because 
of  the  constraint  in  Equation  (3.4),  pk  =  1/r,  proving  that  a  uniform  distribution 
maximizes  entropy  when  no  prior  information  is  available. 

Now  by  adding  prior  information,  such  as  the  feasible  transitions  in  Table  3.1  on 
page  26,  the  distributions  with  maximum  entropy  are  those  that  are  uniform  across 
these  feasible  transitions  .  As  a  result,  the  one  step  transition  probability  matrix  is 
represented  as 

ND  M  F  MF  K 

ND  i/s  Vs  Vs  Vs  Vs 

p  _  M  0  Vs  0  Vs  Vs 

F  0  0  Vs  V3  V3 

MF  0  0  0  1/2  y2 

K  0  0  0  0  1 

which  is  stochastic  and  time  homogeneous  because  {Xn,n  >  0}  is  a  DTMC.  At  the 
beginning  of  an  engagement,  the  target  is  assumed  to  have  kill  state  ND  and  as  such 


a<°)  = 


1  0  0  0  0 


With  the  DTMC  fully  characterized,  any  may  be  calculated  by  transient  analysis. 


3.4  Updating  a^  Using  Bayesian  Inference 

Now  suppose  a1")  is  conditioned  on  situational  parameters.  Using  Bayesian  In¬ 
ference,  an  updated  distribuiton  vector,  a'(n\  can  be  calculated  using  the  information 
present  at  the  time  of  the  assessment. 

Bayes  Theorem  can  be  developed  easily,  using  the  definition  of  conditional  prob¬ 
ability.  In  general 


P{A  n  B}  =  P{A\B}P{B}  =  P{B\A}P{A} 


Simple  algebra  yields  Bayes  Theorem. 

P[AlB} = mmn 

The  Law  of  Total  Probability  states  that  for  a  set  {Aj  :  j  =  1,2,...}  that  are 
mutually  exclusive  and  collectively  exhaustive 


P{B)  =  YJP{B\Aj}P{Aj}. 

j 

Combining  the  Law  of  Total  Probability  with  Bayes  Theorem  yields 


P{Ai\B} 


P{P|A}P{^} 

£P{P|^}P{^} 


producing  the  ability  to  compute  the  posterior  probability  P{Ai\B}  for  all  i  e  J. 

For  a  more  in  depth  review  of  Bayes’  Theorem  or  the  Law  of  Total  Probability, 
the  reader  is  directed  to  [20]. 


3-4-1  Calculation  of  the  Updated  Vector,  a'  .  Let  Bn  denote  that  a  partic¬ 
ular  situation  (i.e.  intersection  of  events)  occurred  for  the  nth  shot  and  a,(n>  denote 
the  distribution  of  {Xn\Bn}.  Now,  the  event  sets  for  the  nth  shot  can  be  represented 
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as 


Movement  =  Mnb 
Engagement  =  Enb 
Activity  =  Anb 
Turret  =  Tnb 
Hull  =  Hnb 
Tracks  =  Trnb 

where  Mnb  denotes  the  occurrence  of  b  within  the  Movement  event  set  for  shot  n.  For 
example,  if  a  soldier  perceives  that  a  target  that  is  moving  after  the  first  shot  it  is 
represented  Miyes.  So 

Bn  =  {Mnb  n  Enb  n  Anb  n  Tnb  n  Hnb  n  Trnb}. 

For  the  update  P{Bn }  must  be  calculated.  There  are  several  ways  to  deal  with 
this  intersection  of  events  which  will  be  discussed  further.  For  now,  assume  that 
P{Bn},  or  more  specifically  P{Bn\Xn},  can  be  calculated.  So  the  calculation  for 
some  a  v  ’  is 

a/(n)  =  [  P{Xn  =  ND|£?„}  . . .  P{Xn  =  K| Bn) 

P{Bn|Xn=ND}P{.Yn=ND|  P{Bn\Xn=K}P{Xn=K} 

E  P{Bn\Xn=j}P{Xn=j}  ■■■  E  P{Bn\Xn=j}P{Xn=j} 
jeE  jeE 

To  develop  a!^n\  the  conditional  event  probabilities,  P{Bn\Xn  =  ND},  must  be  ob¬ 
tained  from  the  survey  data. 

3.4-2  The  Calculation  of  P{Bn}.  The  calculation  of  P{Bn }  is  central  to 
updating  a,(~n\  so  the  method  used  to  calculate  P{Bn }  is  of  utmost  importance. 
Making  no  assumptions,  any  intersection  probability  can  be  calculated  by  the  chain 
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rule  as 


P{A  nBnC}  =  p{A\b  n  C}P{B\C}P{C}. 

Now,  Bn  is  the  intersection  of  six  separate  events  so  calculating  P{Bn}  with  no 
assumptions  makes  the  problem  somewhat  intractable.  However,  several  types  of 
assumptions  could  be  made  to  ease  the  burden  of  calculating  many  conditional  prob¬ 
abilities. 

The  most  simplifying  assumption  would  be  to  assume  that  each  event  set  is 
independent  of  the  others.  This  implies  that  the  occurrence  within  the  Movement 
event  set  does  not  affect  the  probabilities  of  events  within  Engagement,  Turret,  or 
any  of  the  other  event  sets.  As  a  result,  the  probability  of  a  situation  can  be  calculated 

=  P{Mnb}P{Erlb]P{Anb}P{Tnb]P{Hnb}P{Trnb) 

=  II  (*)«»}  U7) 

(*)G  Bn 

where  (*)nb  is  each  respective  event  occurrence  in  the  situation  Bn. 

Realistically,  the  assumption  of  independence  among  all  event  sets  does  not 
make  sense.  The  likelihood  of  certain  actions  will  be  affected  by  physical  damage. 
For  example,  a  soldier  is  more  likely  to  perceive  a  target  moving,  given  he  perceived 
the  tracks  to  have  no  damage.  So  to  avoid  extensive  calculations,  a  set  of  assumptions 
will  be  made 

1.  Event  sets  pertaining  to  Physical  Damage  are  independent  of  each  other. 

2.  Each  event  set  in  Actions  is  dependent  on  all  the  Physical  Damage  event  sets. 

3.  Event  sets  pertaining  to  Actions  are  conditionally  independent  of  each  other 
given  the  physical  damage  present. 
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The  above  assumptions  allow  the  probability  of  a  situation  to  be  calculated  as 


P{Bn}  =  p{m  nEnAnrnHnTr} 

=  P{M  ni?n  A\T  nffn  Tr}P{T  nHn  Tr} 

=  P{M  n  E  n  A\T  n  H  n  Tr}P{T}P{H}P{Tr} 

=  P{M\T  nHn  Tr}P{E\T  nHn  Tr}P{A\T  nHn  Tr}P{T  }P{H}P{Tr} 

(3.8) 

where  subscripts  are  excluded  for  brevity.  The  conditional  probabilities  needed  for 
Equation  (3.8)  can  be  calculated  easily  from  the  survey  data. 

3-4-3  Numerical  Representation  of  the  Data.  Using  the  data  available, 
probabilities  of  observing  events  are  determined  given  a  target’s  current  kill  state. 
The  first  task  is  to  assign  probability  distributions  to  each  of  the  event  sets  for  all 
singular  SMEs  depending  on  his  completed  response  tables.  Mutual  exclusivity  and 
collective  exhaustiveness  of  the  events  within  each  set  allow  us  to  develop  numerical 
distributions.  Assigning  numerical  values  to  responses  marked  Always  and  Never  (0 
and  1,  respectively)  is  trivial.  However,  for  those  event  sets  where  the  respondent 
deems  all  or  some  of  the  events  possible  (i.e.  Might),  a  different  situation  arises. 
Given  that  each  event  in  this  situation  is  marked  similarly  as  Might ,  the  assignment 
of  equal  probabilities  to  each  is  the  most  logical  decision  by  the  principle  of  maximum 
entropy. 

As  an  example,  if  a  soldier  marked  the  Activity  event  set  with  the  responses 
displayed  in  Table  3.2  then  the  event  Seeking  Cover  is  assigned  0  probability,  while 

Table  3.2:  Example  Event  Set  Response 


Event 

Response 

Seeking  Cover 
Taking  Firing  Position 
Personnel  Abandoning 
Other/No  Actions 

Never 

Might 

Might 

Might 

32 


Taking  a  Firing  Position,  Personnel  Abandoning,  and  Other/No  Actions  would  each 
have  a  1/3  probability.  This  inherently  gives  a  individual’s  distribution  of  an  event 
set. 

Now,  using  each  individual’s  responses,  a  distribution  representative  of  all  the 
survey  participants  can  be  developed.  The  calculation  is  simply  an  average  across  all 
individual  distributions. 

To  illustrate,  let  Table  3.2  on  the  previous  page  be  one  soldier’s  response  and 
Table  3.3  be  another  soldiers  response.  This  gives  two  distribution  vectors,  say  di 


Table  3.3:  Additional  Example  Event  Set  Response 


Event 

Response 

Seeking  Cover 
Taking  Firing  Position 
Personnel  Abandoning 
Other/No  Actions 

Alight 

Alight 

Never 

Alight 

and  d2  respectively. 


di 


0  1/3  1/3  1/3 


d2 


1/3  1/3  0  1/3 


Averaging  di  and  d2  yields  the  population  distribution 


d 


1/6  1/3  1/6  1/3 


for  the  Activity  event  set. 

This  method  can  easily  be  extended  across  all  event  sets  for  multiple  survey 
responses.  Now  let  d^j,  r/  e  {1,2,  ...,N},  denote  the  vector  obtained  for  the  rfh 
survey  response  for  the  ith  kill  state,  and  dj  be  the  average  of  the  N  vectors 

1  N 

d-  =  NZ>»’‘ 

r;=l 
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.  So  the  distributions  of  all  event  sets  for  each  kill  state  can  be  calculated  in  this 
manner,  and  used  to  determined  the  P{Bn\Xn  =  j}s  in  Equation  (3.6). 


3.5  Integration  with  Combat  Models 


Modeling  the  BDA  process  as  a  DTMC  and  updating  the  distribution  vectors 
by  Bayesian  inference  have  been  shown  mathematically  in  the  previous  section.  Now 
a  procedure  to  integrate  the  mathematical  representation  into  a  combat  model  will 
be  developed. 


The  combat  models  that  concern  this  research  use  decision  tables.  These  deci¬ 
sion  tables  use  the  current  values  of  several  parameters  in  a  logical  flow  to  determine 
the  distribution  that  an  entity’s  next  action  should  be  drawn  from.  In  this  light,  the 
logic  flow  for  the  current  representation  of  the  BDA  process  within  an  engagement  is 
shown  in  Figure  3.2. 


Figure  3.2:  Method  Integration  Logic  Flow 
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The  combat  model  enters  the  process  upon  an  entity  engaging  a  target  which 
proceeds  to  fire  a  shot  at  the  target  in  node  A.  From  there,  two  streams  of  information 
split  off  and  develop  in  parallel.  Nodes  B,  C,  and  D  are  the  brer’s  perception  and 
nodes  G  and  H  represent  the  target’s  ground  truth.  From  the  ground  truth  side,  the 
targets  damage  and  true  kill  state  are  determined  through  PH  and  PK  calculations 
in  node  G.  Node  H  uses  the  kill  state  to  generate  actions  and  physical  damage  for  the 
target.  From  the  brer’s  perspective,  node  B  takes  all  the  information  from  any  prior 
shots  and  calculates  the  a ^  vector.  The  brer  then  decides  decides  whether  situational 
parameter  data  is  available  for  this  shot  at  node  C.  If  it  is,  the  engaging  entity  uses  its 
sensors  to  gather  actions  and  physical  damage  from  the  target  at  node  D  (generated 
at  node  H).  From  these  situational  parameters,  an  updated  a'  n>  is  calculated.  Now, 
at  node  E  the  brer  makes  a  determination  of  the  kill  state  by  drawing  a  random 
number  and  using  either  a*™)  or  a/(n-1  as  a  BDA  distribution.  Finally  at  node  F,  if 
the  perceived  BDA  kill  state  is  more  than  or  equal  to  the  desired  BDA  kill  state, 
the  model  exits  the  heuristic.  Otherwise,  the  brer  returns  to  node  A  and  repeats  the 
process. 

3. 6  Example  Calculations 

To  illustrate  the  data  development,  transient  analysis  of  the  DTMC,  and  up¬ 
dating  using  Bayesian  Inference,  example  calculations  will  be  shown  in  the  context  of 
the  logic  shown  in  Figure  3.2  on  the  preceding  page.  For  the  purposes  of  simplicity, 
only  the  event  sets  pertaining  to  Actions  (i.e.  Movement,  Engagement,  and  Activity ) 
will  be  considered  so  that  mutual  independence  can  be  assumed.  Also,  for  the  sake 
of  brevity,  the  data  calculations  are  based  on  bve  survey  responses.  The  scenario  will 
consist  of  a  Blue  tank  engaging  a  Red  tank. 

Survey  responses  were  simulated  for  bve  soldiers.  As  an  example,  the  responses 
for  a  tank  in  kill  state  ND  are  shown  in  Table  3.4  on  the  next  page.  The  responses 
are  coded  A  for  events  the  soldier  would  always  see,  M  for  those  he  might  see,  and  N 
for  events  he  would  never  see. 
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Table  3.4:  Simulated  Survey  Responses  for  Tank  at  No  Damage 

Response  Number 


Event  Set 

Event 

1 

2 

3 

4 

5 

Movement 

Yes 

M 

M 

A 

N 

M 

(M) 

No 

M 

M 

N 

A 

M 

Engagement 

Yes 

M 

M 

M 

N 

M 

(E) 

No 

M 

M 

M 

A 

M 

Activity 

Seeking  Cover 

N 

M 

M 

M 

M 

(A) 

Taking  Firing  Position 

M 

M 

M 

M 

M 

Personnel  Abandoning 

N 

M 

N 

N 

N 

Other/No  Actions 

M 

M 

M 

M 

M 

From  the  responses,  distributions  (d^D)  are  developed  for  each  soldier  and 
averaged  across  the  five  responses  giving  And  following  the  method  described  in 
Section  3.4.3.  The  distributions  created  from  the  data  in  Table  3.4  are  displayed  in 
Table  3.5.  Each  of  the  table  entries  pertain  to  an  element  of  d^ND-  For  example 


Table  3.5:  Probabilities  Obtained  from  Simulated  Data  in  Table  3.4 

Response  Number  (77) 


Event  Set 

Event 

1 

2 

3 

4 

5 

d^VD 

Movement 

Yes 

0.5 

0.5 

1.0 

0.0 

0.5 

0.5 

(M) 

No 

0.5 

0.5 

0.0 

1.0 

0.5 

0.5 

Engagement 

Yes 

0.5 

0.5 

0.5 

0.0 

0.5 

0.4 

(E) 

No 

0.5 

0.5 

0.5 

1.0 

0.5 

0.6 

Activity 

Seeking  Cover 

0.0 

0.25 

0.33 

0.33 

0.33 

0.25 

{A) 

Taking  Firing  Position 

0.5 

0.25 

0.33 

0.33 

0.33 

0.35 

Personnel  Abandoning 

0.0 

0.25 

0.0 

0.0 

0.0 

0.05 

Other /No  Actions 

0.5 

0.25 

0.33 

0.33 

0.33 

0.35 

dyND  (- A-SeekingCov  er)  0. 

The  probabilities  displayed  in  the  And  column  are  the  conditional  distributions 
of  the  events  given  a  kill  state  of  ND.  The  conditional  probabilities  for  the  remaining 
kill  states  are  calculated  in  the  same  manner  as  the  ND  case  and  given  in  Table  3.6  on 
the  next  page. 

So,  imagine  a  blue  tank  deems  a  red  tank  as  a  viable  target  and  decides  to 
shoot  at  it  (Node  A).  As  previously  stated,  without  prior  knowledge,  the  red  tank  is 
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Table  3.6:  Event  Probability  Vectors  for  Simulated  Data 


Event  Set 

Event 

djV_D 

d  m 

d^ 

d  mf 

dp 

Movement 

Yes 

0.500 

0.000 

0.700 

0.000 

0.000 

(M) 

No 

0.500 

1.000 

0.300 

1.000 

1.000 

Engagement 

Yes 

0.400 

0.600 

0.000 

0.000 

0.000 

(E) 

No 

0.600 

0.400 

1.000 

1.000 

1.000 

Activity 

Seeking  Cover 

0.250 

0.066 

0.300 

0.066 

0.100 

(A) 

Taking  Firing  Position 

0.350 

0.367 

0.167 

0.000 

0.000 

Personnel  Abandoning 

0.050 

0.367 

0.233 

0.567 

0.000 

Other/No  Actions 

0.350 

0.200 

0.300 

0.367 

0.900 

assumed  to  have  an  ND  kill  state  before  the  engagement,  so 


a 


(o) 


1  0  0  0  0 


and  as  a  result  (at  node  B) 

a(1)  =  a(«)p 

=  0.2  0.2  0.2  0.2  0.2  . 

After  the  shot,  the  target  is  assigned  damage  and  a  kill  state  (node  G).  Con¬ 
sequently  actions  and  physical  damage  are  determined  by  the  model  (node  H).  The 
intersection  of  these  events  makes  up  the  situation,  B\.  In  this  case 


{  El N  o  61  Eyes  61  Ay  aking  Firing  Position}  ■ 


If  the  Blue  tank  can  perceive  the  situational  parameters  (node  C)  then  a/(n'1  can 
be  calculated  (node  D).  First,  calculate  the  P{B\\Xi  =  i}s  for  i  £  S  by  simply  mul¬ 
tiplying  the  appropriate  conditional  probabilities  (from  Table  3.6)  together  according 
to  Equation  (3.7).  The  row  entitled  P{B i|Ad  =  i]  in  Table  3.7  on  the  following  page 
shows  these  intersection  probabilities. 
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Table  3.7:  Calculations  for  a'1^ 

Kill  State  ( i ) 


Event  Set 

Event 

ND 

M-kill 

F-kill 

MF-kill 

K-kill 

Movement 

No 

0.500 

1.000 

0.300 

1.000 

1.000 

Engagement 

Yes 

0.400 

0.600 

0.000 

0.000 

0.000 

Activity 

Other/No  Actions 

0.350 

0.200 

0.300 

0.367 

0.900 

0.0778 

0.0066 

0.0000 

0.0000 

0.0000 

a7« 

0.9222 

0.0778 

0.0000 

0.0000 

0.0000 

Now,  using  the  law  of  total  probability,  find  P{B{\  by  summing  the  conditional 
situation  probabilities  multiplied  by  its  respective  kill  state  probability. 


Y,P{Bi\Xl=j}P{X1=j} 

3&S 


(0.0778)(0.2)  +  (0.0066)(0.2)  +  (0)(0.2) 
+  (0)(0.2)  +  (0)(0.2) 

0.0169 


Finally,  calculate  the  a7^  using  Equation  (3.6).  Equation  (3.9)  demonstrates  example 
calculations  for  the  probability  of  an  ND  kill  state  given  the  situation,  and  Table  3.7 
lists  all  of  the  a7^  vector  probabilities. 


P{X\  =  NDlfix} 


P{B  i|Ad  =  ND}P{X!  =  ND} 

PW} 

(0.0778)(0.2) 

0.0169 


=  0.9222 


(3.9) 


At  this  point  (node  E),  the  blue  tank  gunner  has  0.9222  and  0.0778  probabilities 
of  determining  the  red  tank  to  be  in  the  ND  or  M  kill  states,  respectively.  However, 
there  is  zero  probability  that  the  target  has  F,  MF,  or  K  kill  states.  Assume  a 
random  draw  by  the  combat  model  yields  the  perceived  BDA  as  {X\  —  M } .  This 
would  represent  an  overestimation  of  battle  damage. 
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If  the  desired  BDA  is  an  M-kill  (node  F)  the  Blue  tank  would  exit  the  heuristic. 
If  not,  the  gunner  fires  another  shot  at  the  red  tank  (return  to  node  A)  and  the  process 
is  repeated.  This  time,  however,  the  initial  distribution  before  the  second  shot,  a'^, 
is  known,  so  a1-2^  is  calculated  accordingly. 

a ^  =  a,(1)P 


This  process  is  repeated  until  the  Blue  tank  perceives  adequate  BDA  for  its  target  or 
the  number  of  shots  exceeds  the  engagement  limit. 

3.6.1  Calculating  Conditional  Probabilities.  Now  if  all  event  sets  were  used 
for  the  above  example,  calculating  the  P{Bn\Xn  =  i}s  would  be  accomplished  using 
the  assumptions  in  3.4.2  and  the  appropriate  conditional  probabilities  needed  for  this 
situation  would  be  dictated  by  the  physical  damage  observed  in  the  situation. 

The  probabilities  for  the  Physical  Damage  event  sets  are  given  in  Table  3.8 
These  probabilities  are  needed  explicitly  in  Equation  (3.8).  Now  suppose  for  the 


Table  3.8:  Event  Probability  Vectors  for  Simulated  Data 


Event  Set 

Event 

d  nd 

d  m 

d^ 

d  mf 

dK 

Turret 

No  Damage 

0.725 

0.717 

0.133 

0.033 

0.100 

m 

Light  Damage 

0.275 

0.267 

0.433 

0.408 

0.425 

Heavy  Damage 

0.000 

0.017 

0.433 

0.558 

0.475 

Hull 

No  Damage 

0.408 

0.383 

0.342 

0.358 

0.000 

(H) 

Light  Damage 

0.358 

0.333 

0.342 

0.358 

0.450 

Heavy  Damage 

0.233 

0.283 

0.317 

0.283 

0.550 

Tracks 

No  Damage 

0.792 

0.083 

0.688 

0.050 

0.067 

(Tr) 

Light  Damage 

0.192 

0.408 

0.288 

0.425 

0.467 

Heavy  Damage 

0.017 

0.508 

0.013 

0.525 

0.467 

previous  example,  B\  is  instead 


B, 


■■  {mNo  n  e  Yes  P  ^-Taking  Firing  Position  P  Tj\j0f)arnage  Pi  HNoDamage  P  Tv  LightDamage}  • 


39 


As  a  result,  the  appropriate  conditional  probabilities  needed  to  obtain  P{Bi\i} 


are 


-l/.Yo  Id  H  jy  q  Darn  a  g  e  d  T  V  LightDamageJ  i 

P \Ey  es\P N  oDamage  bl  E  N  oT)  amage  Id  P T  Light  Damage  }  5  and 


P {-^-Taking Firing Position\E]\f oDamage  Id  E J\ o f) a raa g e  d  Tv  Light  Damage  } 


for  each  kill  state.  Recall  that  the  survey  responses  are  represented  in  the  form  of 
d?M;  vectors  containing  the  rfh  soldier’s  event  probabilities  for  kill  state  i.  In  general, 
conditional  probabilities  can  be  calculated  from  data  in  this  form  by 


P{C\D} 


N 

E  P{C},P{D], 


U=1 


N 

E  P{D }„ 


V= 1 


N 


£  dv,i(C)dV}i(D) 
V=1 


N 


E  d„i (D) 

U=1 


(3.10) 


where  C  and  D  are  any  event. 

As  an  example,  P{MNo\TrLightDamage}  will  be  calculated  for  the  ND  kill  state. 
Table  3.9  on  the  following  page  is  an  extension  of  Table  3.5  used  in  the  above  example. 
Recall  that  the  table  entries  pertain  to  elements  of  d^ND- 

Now,  following  Equation  (3.10) 


P  {MNo\Tr  Light,  d  amage  j 


drj,i(MN0)dri,i(T'rLightD  amage) 

T]=l 

5 

^T]  ,2  Light  Damage) 

T)=l 

(0.5)(0)  +  (0.5)(0.5)  +  (0)(0.5)  +  (1)(0.5)  +  (0.5)(0) 
0  +  0.5 +  0.5 +  0.5  +  0 


=  0.5 
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Table  3.9:  Probabilities  for  Individual  Soldiers 

Response  Number  (■ rj ) 


Event  Set 

Event 

1 

2 

3 

4 

5 

Movement 

Yes 

0.5 

0.5 

1 

0 

0.5 

(M) 

No 

0.5 

0.5 

0 

1 

0.5 

Engagement 

Yes 

0.5 

0.5 

0.5 

0 

0.5 

(E) 

No 

0.5 

0.5 

0.5 

1 

0.5 

Activity 

Seeking  Cover 

0 

0.25 

0.333 

0.333 

0.333 

(A) 

Taking  Firing  Position 

0.5 

0.25 

0.333 

0.333 

0.333 

Personnel  Abandoning 

0 

0.25 

0.000 

0.000 

0.000 

Other/No  Actions 

0.5 

0.25 

0.333 

0.333 

0.333 

Turret 

No  Damage 

0.5 

0.5 

1 

0.5 

1 

CO 

Light  Damage 

0.5 

0.5 

0 

0.5 

0 

Heavy  Damage 

0 

0 

0 

0 

0 

Hull 

No  Damage 

0.5 

0.5 

0.5 

0.5 

0.5 

(H) 

Light  Damage 

0.5 

0.5 

0 

0.5 

0.5 

Heavy  Damage 

0 

0 

0.5 

0 

0 

Tracks 

No  Damage 

1 

0.5 

0.5 

0.5 

1 

( Tr) 

Light  Damage 

0 

0.5 

0.5 

0.5 

0 

Heavy  Damage 

0 

0 

0 

0 

0 

so  in  this  very  simplistic  case  P{Mm0}  does  not  change  by  adding  the  condition.  With 
more  survey  responses,  however,  the  probability  will  likely  change  according  to  the 
physical  damage  present. 

Recall  the  situation  in  this  example 


R,  =  f  M, 


Following  the  above  procedure  P{Bi}  for  each  kill  state  and  the  resulting  a'^  vec¬ 
tor  are  displayed  in  Table  3.10.  Adding  the  physical  damage  conditions  drastically 

Table  3.10:  P{Bi}  and  Using  Physical  Damage  Conditions 

Kill  State  ( i ) 


ND 

M-Kill 

F-Kill 

MF-Kill 

K-Kill 

a''1’ 

0.0439 

0.1731 

0.2100 

0.8269 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 

0.0000 
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changes  the  probabilities  in  a/(1).  In  the  first  example  the  Bine  tank  gunner  had 
0.9222  and  0.0778  probabilities  of  determining  the  red  tank  to  be  in  the  ND  or  M  kill 
states,  respectively.  Now  by  adding  the  physical  damage  conditions,  the  probabilities 
are  0.1731  and  0.8269  for  ND  and  M.  So,  it  is  easy  to  see  why  the  dependence  of 
perceived  actions  on  the  perceived  physical  damage  is  important. 

3. 7  Conclusion 

The  BDA  process  in  real  time  is  a  very  complex  and  as  such  has  not  been  well 
modeled  in  the  past.  This  chapter  presents  a  top  to  bottom  approach  in  developing  an 
adequate  BDA  representation  for  direct  engagement  models.  First,  a  method  to  collect 
the  adequate  data  for  real  time  BDA  in  close  combat  was  proposed.  Representation 
of  the  BDA  process  as  a  Markov  Chain  was  developed  and  a  methodology  to  update 
the  nth  shot  distribution  vector  was  discussed.  Next,  a  strategy  to  implement  the 
methodology  into  a  combat  simulation  model  was  introduced.  The  chapter  concludes 
by  providing  calculations  for  a  simple  example  scenario. 
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IV.  Results  and  Analysis 


4-1  Introduction 

Modeling  the  BDA  process  as  a  DTMC  with  updates  by  Bayesian  inference  is 
an  effective  way  to  capture  its  complicated  nature  in  real  time.  The  most  important 
issue  with  this  model  is  that  the  updates  are  entirely  data  dependent.  The  updates  are 
performed  directly  using  the  conditional  probabilities  computed  from  survey  responses 
and,  as  a  result,  are  only  useful  if  the  probabilities  are  (relatively)  representative  of 
the  population. 

This  chapter  will  first  present  an  analysis  of  the  number  of  surveys  collected 
(N)  and  how  it  affects  singular  event  probabilities  as  well  as  the  properties  of  the 
updated  a/("nd  From  this  analysis  a  sample  size  will  be  suggested  for  an  example 
implementation  of  the  methodology.  The  example  analysis  will  include  calculations 
of  the  event  probabilities  and  the  update  to  a/(n)  for  a  three  shot  engagement  sequence 
and  consider  a  case  where  situational  information  is  unavailable. 

4-2  Effects  due  to  the  Number  of  Survey  Responses 

The  conditioning  of  some  distribution  vector  a1'"'-*,  to  give  a/(-n\  depends  directly 
on  the  survey  data  obtained.  This  dependence  stems  from  computing  the  conditional 
event  probabilities  (i.e.  probabilities  of  events  given  a  kill  state)  from  the  survey 
responses  and  using  Bayes’  Theorem  to  update  the  distribution  vector.  It  follows 
that  the  updated  distribution  will  vary  with  the  data  collected. 

The  survey  responses  can  be  thought  of  as  a  sample  of  N  from  the  population  of 
all  soldiers.  Calculating  the  probabilities  of  events  for  Physical  Damage  and  Actions 
is  simply  an  average  across  all  N  responses  to  obtain 


u=i 
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The  (strong)  law  of  large  numbers  states  that,  for  any  sequence  of  random 
independently  and  identically  distributed  (iid)  random  variables  Yv,  77  e  { 1,2, ,  N} 
with  known  and  finite  mean  /iy 

p{  lim  YN  =  pY}  =  l  (4.1) 

IN— >00  ) 


where 

1  N 
r;=l 

So  the  analogous  case  here  can  be  stated 

P  \  lim  d N  i  =  d p  i  \  =  1 

IN— KX)  J 

where  dMji  is  the  vector  of  true  event  probabilities  given  a  kill  state  i.  By  extension, 
the  updated  probability  distribution  of  BDA  estimated  from  N  survey  responses,  a l(jl\ 
will  also  approach  its  true  values  as  N  approaches  oo. 

The  intuitive  result  here  is  that  more  survey  responses  will  yield  better  estimates 
of  d and  subsequently  However  a  practical  target  number  of  survey  responses 
required  for  the  modeling  methodology  is  of  great  interest.  Since  both  dj  and  the 
resulting  a /(n)  are  easily  observable,  this  issue  can  be  handled  from  two  angles.  Both 
estimates  will  be  explored  via  simulation. 

4-2.1  Simulation  of  Survey  Responses.  To  suggest  a  target  number  of  survey 
responses  (N),  data  must  be  produced  to  give  insight  into  an  adequate  number  of 
surveys  for  which  the  model  will  be  effective.  As  such,  a  simulation  technique  was 
developed  for  this  purpose. 

To  simulate  survey  responses,  underlying  probabilities  were  set  to  compare  ran¬ 
dom  numbers  against.  Within  each  Physical  Damage  event  set,  probabilities  of  a 
respondent  marking  events  as  possible  were  determined  to  give  intuitive  proportions 
of  survey  responses.  Probabilities  of  marking  events  in  the  Action  event  sets  were 
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conditioned  based  on  how  the  Physical  Damage  portion  is  filled  out.  The  data  simu¬ 
lated  is  meant  to  be  intuitive  (e.g.  P{Myes \Xn  =  M}  will  be  small)  to  lend  credibility 
to  the  analysis. 

Survey  responses  were  simulated  using  the  Matlab®  programming  language. 
First,  a  vector  of  17  random  numbers  was  generated.  The  physical  damage  portion 
of  the  survey  is  filled  by  comparing  the  random  numbers  to  the  probabilities  of  the 
respondent  marking  an  event  possible.  The  actions  are  filled  depending  on  the  re¬ 
sponse  to  the  Physical  Damage  event  sets.  The  result  is  a  vector  of  1  s  and  Os  which 
represent  the  respondent  marking  events  as  possible. 

The  binary  vector  is  transformed  into  Ns,  Ms,  and  As  and  further  into  event 
probabilities  (d^s)  by  placing  equal  probability  into  any  events  marked  possible.  An 
event  set  with  only  one  event  marked  possible  is  assigned  probability  of  1.  If  two 
events  are  marked  possible,  each  receives  0.5  and  so  on  following  the  principle  of 
maximum  entropy.  The  case  where  an  event  set  is  null  (no  events  marked  possible) 
results  in  a  random  draw  among  the  possible  distributions. 

The  set  of  N  total  survey  responses  is  used  to  estimate  the  probabilities  of 
events  given  physical  damage  and  further  in  updating  a ^  to  a,<nh  The  Matlab® 
code  used  to  simulate  these  responses,  perform  calculations,  and  plot  data  can  be 
found  in  Appendix  B. 

4-2.2  Effect  of  N  on  the  Event  Probabilities.  The  conversion  of  survey 
responses  into  event  probabilities  and  their  direct  contribution  to  the  calculations  of 
an  update  has  been  discussed.  Recall  that  the  event  probabilities  contained  in  dj  will 
approach  their  true  values  as  the  number  of  survey  responses  N  approaches  oo. 

The  Central  Limit  Theorem  (CLT)  implies  that  a  standardized  sum  of  random 
variables  will  approach  a  standard  normal  distribution.  For  the  sequence  of  iid  random 
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variables,  Yv,  77  =  1,  2, ,  N,  with  known  fiy  and  ay 


lim  P  \  <  4  =  «W  (4.2) 

l^v'/VN  j 

where  denotes  the  sample  mean,  and  $>(z)  is  the  value  of  the  standard  normal 
cumulative  distribution  function  at  z. 

As  a  result,  the  elements  of  dj(*)  can  be  treated  with  normal  distribution  theory. 
One  result  is  that  the  event  probabilities  determined  by  averaging  survey  data  will 
approach  the  true  values  because  the  standard  error  of  any  Yy  has  the  property 


lim  —  =  lim  ay  —  0. 

N^oo  N  N— >00 


(4.3) 


As  an  example,  the  true  distribution  of  individual  responses  for  dr;,ND  (TNoDamage) 
(i.e.  P {TNoDamage \Xn  =  ND})  is  known  because  the  survey  responses  are  simulated. 
This  distribution  is 


-Pjd^ND  (TNoDamage)  —  *}  — 


( 


*  =  0  : 

w.p. 

0.0310 

*  =  1/3  : 

w.p. 

0.0285 

*  =  1/2: 

w.p. 

0.5605 

*  =  1  : 

w.p. 

0.3800 

\ 


with  known  nd  n  )  —  0.6695  and  al  (T  —  0.0747.  Now, 

NoDamage)  ^77, ND  \-L  NoDamage )  ’ 

4nd (TNoDamage)  —>  0.6695  as  the  number  of  survey  responses,  N  — >  oo  by  Equation 
(4.1).  Figure  4.1  on  the  following  page  plots  the  standard  error  of  dND(T/vD.Da»nage) 
for  this  distribution  against  numbers  of  survey  responses  and  displays  the  behavior 
predicted  in  Equation  (4.3).  Technically  a  target  number  of  survey  responses  could 
be  calculated  for  a  specified  bound(s)  on  the  estimate  d NT>(TNoDamage)-  Realistically, 
however,  the  distribution  will  not  be  known  in  advance  and  cInd  (TNoDamage)  will  be 
the  only  estimate  available  for  dM)ND  {TNoDamage)  ■ 
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Figure  4.1:  adm(TNoDamage ),  versus  number  of  survey  responses,  N 


A  practical  solution  is  to  simulate  survey  responses  from  this  distribution  and 
observe  the  event  probability  reach  a  relatively  stable  estimate  of  the  true  event  prob¬ 
ability.  Figure  4.2  on  the  next  page  shows  a  continuous  calculation  of  <1nd  (TNoDamage) 
as  N  increases.  Because  the  response  distribution  is  known,  a  comparison  can  be  made 
with  the  true  value  of  /ad  m(TNoDarnage),  0.6695,  denoted  by  the  reference  line. 

This  type  of  simulation  exercise  could  certainly  be  performed  for  each  event 
probability.  However,  the  dependence  of  Actions  on  Physical  Damage  would  indi¬ 
cate  the  presence  of  several  interactions  causing  a  drastic  increase  in  the  number  of 
probabilities  that  need  examination.  As  a  result,  such  an  investigation  would  prove 
impractical.  Because  of  the  dependence,  a  more  efficient  way  to  observe  the  effect  of 
N  on  the  model  is  to  compare  the  estimated  and  true  distributions  of  Xn,  a and 
a'M  respectively. 

4-2.3  Effect  of  N  on  the  Moments  of  a!^n\  A  common  method  to  compare 
two  distributions  is  to  investigate  their  respective  moments.  The  moments  themselves 
do  not  specify  a  distribution,  but  rather,  the  characteristic  function  (a  function  of 
the  moments)  does.  Distributions  known,  or  assumed  to  be  of  the  same  form  (e.g. 
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Figure  4.2:  Simulated  cInd (TNoDamage)  versus  number  of  survey  responses,  N 

nominal  discrete  in  this  case)  may  be  compared  on  the  basis  of  moments  for  this 
reason.  Indeed,  many  hypothesis  tests  do  just  that  by  estimating  parameters  from 
data  to  draw  conclusions  about  the  distribution  of  a  sample. 

The  two  most  commonly  used  moments  of  a  distribution  are  the  mean  (i.e. 
expected  value)  and  the  variance.  The  mean  is  a  raw  moment  (about  the  origin)  and 
signifies  location.  Variance  is  the  second  central  moment  (about  the  mean)  and  is  a 
measure  of  dispersion.  The  third  and  fourth  order  central  moments  measure  skewness 
(symmetry)  and  kurtosis  (peakedness)  respectively. 

The  methodology  is  heavily  dependent  on  data,  so  any  variability  or  biases 
present  in  the  survey  responses  will  be  reflected  in  the  moments  of  a,(”k  BDA  has 
a  nominal  discrete  distribution  with  nominal  classes  and,  as  a  result,  the  concepts 
of  a  (conditional)  expected  value  and  variance  of  Xn,  given  a  situation  (E[Xn\Bn] 
and  Var[Xn\Bn]  respectively)  do  not  have  intuitive  meaning.  The  nominal  nature 
of  the  distribution  means  that  the  kill  states  show  neither  a  proportional  quantity 
(cardinality)  nor  rank  (ordinality).  To  calculate  any  moments  for  the  distribution, 
the  kill  states  must  be  given  a  cardinal  support.  An  example  of  a  discrete  distribution 
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with  a  cardinal  support  would  be  a  standard  six  sided  die,  for  which  E[*]  and  Var[*\ 
can  be  calculated  easily. 

For  the  purposes  of  this  demonstration,  let  the  kill  states  take  the  following 
support. 


ND  =  1 
M  =  2 
F  =  3 
MF  =  4 
K  =  5 

Since  the  mapping  is  completely  arbitrary,  the  expected  value  and  variance  of  a 
(or  a,(n'>)  do  not  have  a  direct  interpretation. 

It  is  well  known  that  for  any  discrete  random  variable  Y  with  a  known  proba¬ 
bility  distribution,  the  expected  value  and  variance  can  be  calculated  as 

£[*1  =  E  «piY  = »} 

y 

V ar[Y]  =  -  E[Y])2P{Y  =  y} 

y 

=  E[Y 2]  -  ( E[Y ])2 


where 

E[Y2}  =  Yjy2P{Y  =  V} 

y 

is  the  second  moment  of  Y a  about  the  origin. 

Recall  that  a approaches  the  true  distribution  for  increasing  N.  A  single  a/(^ 
simultaneously  reflects  the  entire  body  of  survey  responses  for  dependent  and  inde¬ 
pendent  event  probabilities  alike.  Since  the  survey  responses  are  simulated  the  true 
value  of  any  a can  be  calculated  and  compared  against  the  estimated  distribu- 
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tion.  Exploring  every  possible  intersection  of  events  (432)  would  make  the  analysis 
intractable,  so  a  subset  of  situations  (Bns)  was  selected  to  represent  a  cross  section 
of  the  possible  intersections. 

To  see  the  effect  of  the  number  of  responses  on  E[Xn\Bn]  and  Var[Xn\Bn], 
survey  responses  were  simulated  sequentially.  At  each  new  value  of  N,  the  distribution 
after  the  Erst  shot  was  updated  to  give  a/(1)  for  each  of  the  situations. 


Ba  —  \iffjVo  H  E]\f0  Pi  Aother /NoAction  hi  Tn oDamage  hi  H ]\j 0Damage  hi  Tv J\r oDamage  f 
Bb  =  {Myes  n  E  Yes  ^  A-SeekingCover  ^  N  oDamage  ^  B LightD amage  ^  Tt n oDamage\ 

Bq  { M-No  ^  B]\ j0  Pi  Awaking  Firing  Position  ^  Bjy  0][)arnage  ^  H LightD  amage  ^  Bt  £Jigjl^£)arnage  j 

Bd  =  {Myes  H  E  No  hi  ApersonneiAbc i/ndon  hi  T [{ eaVy  Damage  hi  H jg0£)amage  Pi  T  f  [Jg},t  Damage.  } 
Be  {  Thv o  El  Bj\j0  P  A ()/  h e r  / N q  A  Ci i on  hi  T[  j g /,  /  f) a  rn  ag ,  P  H peavyp)amage  P  T  V HeaVyDamagef 


where  B *  represents  Situation  *  and  the  numeric  subscript  (1)  is  omitted  for  brevity. 

For  each  of  the  five  situations,  the  expected  value  and  variance  were  calculated 
analytically  from  a/(1h  Recall,  both  E[Xn\Bn]  and  Var[Xn\Bn]  should  approach  their 
true  values  as  the  number  of  survey  responses  increases  by  law  of  large  numbers. 
Figure  4.3  and  4.4  on  the  following  page  show  the  expected  value,  /i,  and  the  variance, 
d2,  plotted  against  the  number  of  survey  responses  for  Bb  and  Be.  The  reference  lines 
indicate  the  true  values  of  the  respective  moments.  Of  the  Eve  situations  investigated, 
Situation  b  approaches  its  true  values  for  the  smallest  values  of  N  and  Situation  e 
requires  the  largest  values  of  N.  In  Figure  4.3,  Situation  b  approaches  its  true  values 
for  (i  and  a2  to  within  simulation  noise  very  quickly;  just  over  N  =  100  survey 
responses.  Figure  4.4  shows  that  //  and  a2  for  Situation  e  approach  their  reference 
lines  more  slowly.  E[Xn\Be]  and  Var[Xn\Bn]  reach  acceptable  levels  for  N  =  700 
survey  responses.  However,  a  case  could  certainly  be  made  for  N  =  1000+  survey 
responses,  given  that  both  moments  appear  to  further  trend  toward  their  reference 
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Figure  4.3:  fi  and  d2  versus  Survey  Responses,  N  (Situation  b) 


Figure  4.4:  /}  and  d2  versus  Survey  Responses,  N  (Situation  e) 

lines  for  N  >  850.  Situations  a ,  c,  and  d  performed  somewhere  in  between  Situations 
b  and  e.  Sample  data  and  plots  for  all  of  the  situations  may  be  found  in  Appendix  C. 

4-2-4  Effect  of  N  on  a'^ .  Recall  that  the  moments  of  a'^  have  no  direct 
interpretation  because  of  the  nominal  nature  of  the  kill  states.  To  avoid  giving  kill 
states  an  arbitrary  support  one  might  think  of  a'^  as  a  point  in  space,  or  otherwise 
stated 

a/(n)  e  M5. 
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The  true  distribution  also  occupies  a  point  in  M5,  so  the  distributions  may  be  compared 
by  calculating  the  distance  between  the  points  they  represent. 


In  multidimensional  space,  different  forms  of  distance  from  the  origin  can  be 
calculated  by  a  p-norm,  ||  •  ||p  and  represented  as 

\\y\\P  =  (\yi\p +  \y2\p  +  ■■■  +  \yn\p)1/p  p>  l  yeR".  (4.4) 

The  2-norm  is  known  as  the  euclidian  norm 

llyll  =  VW  +  I  z/2 1 2  +  •  •  •  +  \yn\2 

=  Vy^y  (4.5) 

and  can  be  used  to  measure  the  straight  line  distance  between  points  in  space.  In  the 
case  of  a4n)  and  a,(n) 


lia'(1)*4'<‘)||  =  v/(«K  -  aS)2  +  ■  ■  ■  +  (4*1  -  Wr 

=  /Elob  -  af  )2-  (4.6) 

V  ieS 

The  result  of  Equation  (4.6)  inherently  represents  how  different  the  distributions 
are  and  captures  all  of  the  moment  information  into  a  single  value.  Again,  as  N 
becomes  large,  a/<n)  will  approach  the  trne  distribution  and  give  the  result 

lim  ||a/(n)  -a,(n)|l  =  0. 

N— xx) 

So  to  compare  distributions  |  la7^1-*  —  a7!1 1 1 1  was  calculated  for  increasing  values  of 
N.  The  same  set  of  survey  data  and  situations  (a  —  e)  as  above  were  used.  Figure  4.5  on 
the  next  page  display  the  plots  for  Situations  b  and  e,  respectively.  Both  situations 
show  similar  results  to  the  moment  plots.  Situation  b  shows  very  small  values  of 
||a/(A  —  a/(4)||  for  any  N  >  150.  Again,  as  in  the  moment  plots,  Situation  e  displays  a 
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Situation  b  Stiuation  e 


Figure  4.5:  ||a'd)  —  a7*-1 1 1 1  versus  Survey  Responses,  N 

much  larger  difference  of  a/(n)  from  a'd)  and  indicates  at  least  N  =  700  for  acceptable 
results.  Situations  a,  c  and  e  show  results  between  those  obtained  for  Situation  b  and 
e.  Their  plots  of  ||a'd)  —  a'd)]]  versus  N  can  be  seen  in  Appendix  C 

Both  methods  of  comparing  a'd)  to  a result  in  similar  proposed  values  for  N. 
This  similarity  suggests  that  either  method  is  adequate  to  determine  a  target  number 
of  survey  responses. 

4-3  Example  Data  Analysis 

To  fully  illustrate  the  methodology,  a  complete  example  implementation  form 
data  collection  to  model  implementation  will  be  completed.  First  a  data  set  of  sur¬ 
vey  responses  will  be  obtained  (simulated)  with  N  =  700  suggested  in  the  previous 
section.  Next  the  set  of  survey  responses  will  be  converted  to  the  appropriate  event 
probabilities.  Lastly  the  algorithm  will  be  run  for  a  three  shot  engagement  sequence 
within  a  combat  simulation  model  providing  the  necessary  calculations. 

4-3.1  Calculation  of  Conditional  Event  Probabilities.  Now,  the  qualitative 
values,  contained  in  the  N  =  700  survey  responses  must  be  transformed  into  estimates 
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of  the  conditional  event  probabilities,  contained  in  d,.  First,  the  qualitative  values 
(TVs,  Ms,  and  As)  marked  on  the  surveys  must  be  mapped  to  their  respective  event  set 
probabilities.  For  each  event  set  within  a  the  most  likely  distribution  is  derived 
from  the  principle  of  maximum  entropy  giving  df;  j.  Once  all  of  the  d,?iis  are  mapped, 
the  dj  are  calculated  by 

1  N 

V  teS 

r)=l 

The  event  probabilities  for  each  kill  state,  d,M;,  estimated  from  the  N  =  700 
simulated  survey  responses  are  given  in  Table  4.1.  The  probabilities  in  this  table  are 

Table  4.1:  Conditional  Event  Probabilities  for  Simulated  Data 


Kill  State  (i) 


Event  Set 

Event 

ND 

M-kill 

F-kill 

MF-kill 

K-kill 

Movement 

Yes 

0.5064 

0.0764 

0.5071 

0.0307 

0.0693 

(M) 

No 

0.4936 

0.9236 

0.4929 

0.9693 

0.9307 

Engagement 

Yes 

0.5064 

0.5014 

0.0743 

0.0336 

0.0579 

(E) 

No 

0.4936 

0.4986 

0.9257 

0.9664 

0.9421 

Activity 

Seeking  Cover 

0.3575 

0.1030 

0.3983 

0.0675 

0.0696 

(A) 

Taking  Firing  Position 

0.3465 

0.3082 

0.0669 

0.0549 

0.0692 

Personnel  Abandoning 

0.0908 

0.3963 

0.2831 

0.1411 

0.4263 

Other/No  Actions 

0.2051 

0.1925 

0.2517 

0.7365 

0.4349 

Turret 

No  Damage 

0.6750 

0.6590 

0.0879 

0.0167 

0.0448 

(T) 

Light  Damage 

0.3014 

0.3162 

0.4100 

0.3974 

0.4133 

Heavy  Damage 

0.0236 

0.0248 

0.5021 

0.5860 

0.5419 

Hull 

No  Damage 

0.6569 

0.5231 

0.5493 

0.1119 

0.2850 

(H) 

Light  Damage 

0.3176 

0.4045 

0.3700 

0.3498 

0.4014 

Heavy  Damage 

0.0255 

0.0724 

0.0807 

0.5383 

0.3136 

Tracks 

No  Damage 

0.6610 

0.0336 

0.5400 

0.0393 

0.0543 

cm 

Light  Damage 

0.3167 

0.4864 

0.4407 

0.3843 

0.4393 

Heavy  Damage 

0.0224 

0.4800 

0.0193 

0.5764 

0.5064 

the  marginal  probabilities  averaged  across  all  N  soldiers.  The  conditional  probabilities 
for  Actions  given  all  possible  intersections  of  Physical  Damage  are  excluded  here  for 
brevity. 


4-3.2  Three  Shot  Engagement  Sequence.  To  illustrate  the  transient  analysis 
of  the  DTMC  and  updating  using  Bayesian  Inference,  the  implementation  logic  will 
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be  followed  for  a  typical  engagement  in  CASTFOREM:  a  three  shot  sequence  for  a 
blue  (friendly)  tank  bring  upon  a  red  (enemy)  tank. 

So,  imagine  a  blue  gunner  deems  a  red  tank  as  a  viable  target  and  decides  to 
shoot  at  it.  Recall  a-")  denotes  the  probability  vector  of  {Xn,n  >  0}  at  time  n  and 
the  initial  distribution  (without  prior  knowledge)  is 

a(0)  =  1  0  0  0  0 

because  the  red  tank  is  assumed  to  have  an  kill  state  of  ND  before  the  engagement. 
Additionally  recall  P  is  the  transition  probability  matrix 


ND 

ND 

Vs 

M 

Vs 

F 

Vs 

MF  K 

Vs  Vs 

M 

0 

73 

0 

73  lk 

F 

0 

0 

Vs 

73  Vs 

MF 

0 

0 

0 

V2  V2 

K 

0 

0 

0 

0  1 

and  as  a  result 

a{1)  =  a(0)P 

=  0.2  0.2  0.2  0.2  0.2 

that  is,  the  prior  distribution  of  X\. 

At  this  point  the  model  decides  (by  random  number  draw)  that  the  Blue  tank 
gunner  will  perceive  situational  data.  In  parallel,  the  model  has  determined  ground 
truth  kill  state  to  be  ND  (unknown  to  the  Blue  gunner)  via  PH  and  PK  calculations. 
Perceived  actions  and  physical  damage  are  determined  by  the  model  using  a  vector 
of  random  numbers  to  draw  from  the  appropriate  conditional  event  set  distributions. 
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A  simulated  random  vector  yields  the  the  situational  data,  Bi  given  by 


D 


which  the  Blue  tank  gunner  will  perceive. 

The  perceived  events,  along  with  their  respective  probabilities  are  listed  in  Ta¬ 
ble  4.2.  The  probabilities  corresponding  to  the  Actions  in  B\  are  conditioned  on  the 

Table  4.2:  Observed  Events  and  Conditional  Probabilities  (Shot  1) 

Kill  State  ( i ) 


Events 

ND 

M-kill 

F-kill 

MF-kill 

K-kill 

Physical 

T7 

N oDamage 

0.6750 

0.6590 

0.0879 

0.0448 

0.0167 

Damage 

H  N  oDamage 

0.6569 

0.5231 

0.5493 

0.2850 

0.1119 

Tt  LightDamage 

0.3167 

0.4864 

0.4407 

0.4393 

0.3843 

Actions 

Mno 

0.4694 

0.9315 

0.5587 

0.9010 

1.0000 

(Conditioned) 

Eno 

0.4483 

0.4946 

0.8691 

0.9219 

0.8929 

Aother  /  N o  Actions 

0.2174 

0.1911 

0.2444 

0.4625 

0.7500 

P{B1\X1  =  i} 

0.00642 

0.01476 

0.00252 

0.00215 

0.00048 

a'(1) 

0.24379 

0.56043 

0.09582 

0.08173 

0.01822 

settings  of  Physical  Damage. 

First,  the  conditional  probabilities  of  the  situation  P{B\\X\  =  i}  are  calculated 
for  each  kill  state  using  Equation  (3.8).  This  is  simply  the  product  of  the  event 
probabilities  listed  in  Table  4.2.  For  example  calculate  the  probability  of  this  situation 
given  X\  =M 


P{Bl\Xi  =  M)  =  P{M]\j0\T]\?0E)amage  (~1  H NoDamage  n  Tf LightDamage} 

' P {Tj jv o\T]\f oDamage  O  H ]\f 0Damage  0  T'f’Lig/itDamage} 

'T{  Aother / n o Actions  \  T^oDamage  O  H NoDamage  O  Tv LightDamage} 
'  P \  l  X ()  j)u  tti ugt  }  "  P  {  H o  f) arna g e  }  '  P {Tf  LightD  amage  } 

=  (0.9315)  (0.4946)  (0.1911)  (0.6590)  (0.5231)  (0.4864) 

=  0.01476 
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Then  by  the  law  of  total  probability 


i\xi=j}pixi=j} 

j&s 

=  ^P{B1|X1=j}a<1> 

=  (0.00642)(0.2)  +  (0.01476)(0.2)  +  (0.00252)(0.2) 
+  (0.00215)(0.2)  +  (0.00048)(0.2) 

=  0.00527. 


Finally,  calculate  the  a/(^  using  Equation  (3.6).  Equation  (4.7)  demonstrates 
example  calculations  for  the  probability  of  an  M  kill  state  given  the  situation,  and 
Table  4.2  on  the  previous  page  lists  all  of  the  a'*-1^  vector  probabilities. 


P{Xl  =  M|  B,} 


P{Bi\Xi  =  ND}P{Xi  =  ND} 

"  iW 

(0.01476)  (0.2) 

0.02364 


=  0.56043 


(4.7) 


Now  the  model  has  an  updated  distribution  for  X\.  Based  on  the  technique 
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Because  this  is  the  second  shot,  the  updated  distribution  for  the  first  round  is 
known  to  be  so  a1-2)  is  calculated  using  the  information  available. 


a(2)  =  a/(1)P 


(4.8) 


ND  M 

0.24379  0.56043 


F  MF  K 

0.09582  0.08173  0.01822 


ND 

i  / 

M 

i/ 

F 

i/ 

MF  K 

I  /  1  / 

75 

0 

75 

Vs 

/5 

0 

75  75 

73  73 

0 

0 

Vs 

73  73 

0 

0 

0 

72  V2 

0 

0 

0 

0  1 

0.048759  0.235568  0.080701  0.308377  0.326596 


Again,  the  model  decides  that  the  gunner  can  observe  the  situational  data.  The 
model  then  generates  the  events  associated  with  the  perceived  situation,  B2,  from  a 
vector  of  random  numbers.  Now 


B2  I  -My  es  f~l  Ejsro  n  Agee kinqCover  FI  Tf 


and  the  events  are  shown  in  Table  4.3  on  the  following  page  with  their  respective 
probabilities.  The  subsequent  calculations  to  obtain  P{B2 \X2  =  j},  j  &  S  and  a,(2> 
(also  displayed  in  Table  4.3  on  the  next  page)  are  analogous  to  those  of  the  first  shot. 


The  prior  distribution  is  calculated  similarly  to  Equation  (4.8)  as 


a 


(3) 


0.027688  0.042012  0.283387  0.320903  0.32601 


Table  4.3:  Observed  Events  and  Conditional  Probabilities  (Shot  2) 

Kill  State 


Events 

ND 

M-Kill 

F-Kill 

MF-Kill 

K-Kill 

Physical 

T 

±  NoDamage 

0.3014 

0.3162 

0.4100 

0.4133 

0.3974 

Damage 

H NoDamage 

0.6569 

0.5231 

0.5493 

0.2850 

0.1119 

T  T  LightDamage 

0.3167 

0.4864 

0.4407 

0.4393 

0.3843 

Actions 

mNo 

0.4684 

0.0785 

0.5067 

0.0769 

0.0238 

(Conditioned) 

Eno 

0.5107 

0.5026 

0.9254 

0.9520 

0.9667 

A-Other  /  N  o  Actions 

0.3684 

0.1119 

0.3975 

0.0773 

0.0774 

P{B2\X2  =  i} 

0.00553 

0.00036 

0.01850 

0.00029 

0.00003 

a'<2> 

0.13844 

0.04297 

0.76710 

0.04638 

0.00511 

As  in  the  previous  two  shots,  the  model  decides  that  the  Blue  tank  gunner  may 
perceive  situational  data.  Additionally  B3  is  generated  by  the  model. 


=  {Ma 


The  events  in  B%  and  their  respective  conditional  probabilities  are  displayed  in 
Table  4.4.  P{B3}  and  a'®  can  be  calculated  in  the  same  manner  as  for  the  first  and 
second  shots  and  are  also  shown  in  Table  4.4. 

Table  4.4:  Observed  Events  and  Conditional  Probabilities  (Shot  3) 

Kill  State 


Events 

ND 

M-Kill 

F-Kill 

MF-Kill 

K-Kill 

Physical 

T1 

LightDamage 

0.3014 

0.3162 

0.4100 

0.4133 

0.3974 

Damage 

H LightDamage 

0.3176 

0.4045 

0.3700 

0.4014 

0.3498 

TTh  eavy  Damage 

0.0224 

0.4800 

0.0193 

0.5064 

0.5764 

Actions 

Mno 

0.5698 

0.9187 

0.5071 

0.9394 

0.9664 

(Conditioned) 

Eno 

0.6395 

0.5083 

0.9574 

0.9249 

0.9486 

^-Personnel  Abandoning 

0.1105 

0.4078 

0.2465 

0.4153 

0.1377 

P{B3\X3  =  t} 

0.00009 

0.01169 

0.00035 

0.03032 

0.01011 

a'(3) 

0.00018 

0.03607 

0.00728 

0.71437 

0.24210 

4-3.3  Engagement  Sequence  with  missing  information.  The  result  stated 
in  Equation  (3.3)  from  transient  analysis  allow  this  methodology  to  deal  with  cases 
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where  situational  parameters  cannot  be  observed.  Suppose  in  the  above  engagement 
no  perception  data  was  available  to  the  gunner  after  the  second  shot,  but  after  the 
third  shot  he  perceives  situational  information  again. 

The  values  of  P{B:i\X:i  =  i\  do  not  change  for  this  case  but  rather  the  prior 
distribution  shifts.  The  calculations  in  this  case  differ  from  those  in  the  above  sequence 
only  in  the  way  a*-3-*  is  obtained.  With  no  information  gained  after  the  second  shot, 
a(3)  is  calculated  from  the  last  known  distribution  a^1-1  by 

a(3)  =  a/(l)p(2) 

=  f  0.00975  0.08827  0.03665  0.26936  0.59596  1 


to  give  the  prior  distribution  for  Bayes’  Theorem. 

Now  a/(3)  is  calculated  in  the  same  manner  as  above  in  Equation  (4.7)  and  yields 


0.00006  0.06772  0.00084  0.53588  0.39551 


where  the  subscript  (1)  indicates  as  the  last  distribution  updated  with  situational 
parameters.  The  difference  between  a//3')  and  a^3)  reflects  the  information  lost  by  not 
gaining  perceptional  data  on  the  second  shot. 


4-4  Conclusion 

This  chapter  provides  an  analysis  of  the  BDA  enhancement  methodology.  First, 
the  issue  of  how  many  survey  responses  are  required  for  the  methodology  was  dis¬ 
cussed  and  a  recommendation  of  sample  size  determined  by  simulation  using  several 
metrics.  Secondly,  a  full  example  implementation  of  the  methodology  was  accom¬ 
plished  using  a  three  shot  engagement  sequence.  Lastly  an  example  calculation  with 
missing  information  was  completed. 
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V.  Recommendations  and  Future  Research 


5. 1  Introduction 

This  thesis  deals  with  the  complexity  of  the  BDA  process  for  close  combat 
situations  and  the  difficulties  in  modeling  it.  The  research  has  three  objectives: 

1.  Develop  a  new  methodology  to  model  the  BDA  process  which  addresses  the 

inadequacies  of  the  current  heuristic. 

2.  Produce  a  data  collection  method  (i.e  a  survey)  that  will  collect  the  necessary 

data  for  the  modeling  technique. 

3.  Propose  a  technique  to  implement  the  methodology  into  a  combat  model. 

The  first  objective  was  completed  by  modeling  BDA  as  a  stochastic  process, 
more  specifically  as  a  discrete  time  Markov  chain  (DTMC).  The  uncertain  nature  of 
BDA  is  captured  within  the  DTMC  model.  The  information  gained  through  bat¬ 
tlefield  perception  was  modeled  via  Bayesian  inference,  using  the  result  of  transient 
analysis,  a  pmf  kill  state  vector,  as  a  distribution  of  prior  probabilities. 

The  second  objective  was  accomplished  by  designing  a  survey  to  collect  data 
pertinent  to  the  proposed  methodology.  A  set  of  event  trees  was  developed  to  enu¬ 
merate  those  perception  events  that  most  affect  a  determination  of  BDA.  These  trees 
helped  to  frame  the  questions  that  will  be  asked  of  combat  subject  matter  experts. 

The  third  objective  was  completed  by  developing  a  logical  modeling  flow  con¬ 
struct  of  the  engagement  process  and  placing  the  proposed  methodology  within  it. 
Further,  an  analysis  of  how  many  survey  responses  will  be  needed  to  implement 
the  methodology  was  performed.  The  results  of  the  DTMC  with  Bayesian  updating 
suggest  that  the  number  of  survey  responses  should  be  maximized  but  that  several 
hundred  will  give  adequate  results. 

5.2  Modeling  Issues 

There  are  several  modeling  decisions  that  need  discussion  regarding  the  rep¬ 
resentation  of  the  BDA  process  as  a  DTMC  with  Bayesian  updating.  First,  the 
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methodology  as  presented  in  this  research  incorporates  more  detail  than  is  currently 
modeled  by  CASTFOREM  and  the  differences  must  be  considered.  Second,  during 
an  engagement,  the  model  must  make  determinations  of  whether  the  entity  will  shoot 
again  or  exit  the  engagement.  There  are  several  techniques  that  the  model  might  use 
to  deal  with  these  problems. 

5.2.1  Level  of  Detail  in  the  Methodology.  This  methodology  was  designed 
with  CASTFOREM  in  mind.  However,  this  research  has  included  a  greater  level  of 
detail  than  explicitly  modeled  in  CASTFOREM  at  this  time.  This  was  done  to  ensure 
that  the  model  was  flexible  enough  be  effective  as  CASTFOREM  evolves,  and  further 
to  transition  into  the  Army’s  next  generation  combat  simulaiton  model,  Combat  XXI. 
The  flexibility  of  this  methodology  is  one  of  best  attributes,  since  as  much  or  as  little 
information  as  the  user  desires  can  be  considered. 

Currently,  CASTFOREM  models  very  little  in  the  way  of  perception  informa¬ 
tion.  For  the  BDA  enhancement  methodology,  Movement  and  Engagement  are  the 
only  two  event  sets  modeled  by  CASTFOREM  at  this  time.  As  a  result,  the  remain¬ 
ing  event  sets  must  be  dealt  with  in  one  of  several  ways.  First,  the  Activity  event  set 
and  those  pertaining  to  Physical  Damage  might  be  ignored.  Second,  the  event  sets 
not  explicitly  modeled  might  be  selected  by  a  random  number  generator,  based  on 
the  ground  truth  state  of  the  target.  Probabilities  obtained  from  the  survey  responses 
could  be  used  for  this  purpose.  Either  way  will  yield  an  improved  BDA  heuristic  for 
CASTFOREM,  but  the  full  capabilities  of  the  methodology  are  achieved  using  more 
detail. 

5.2.2  Engagement  Exit  Criterion.  The  combat  model’s  decision  to  exit  or 
proceed  with  an  engagement  is  a  very  important  issue.  Overestimation  and  underes¬ 
timation  of  BDA  on  the  battlefield  are  a  result  of  uncertainty  in  the  model  and  have 
effects  on  the  results  of  a  simulated  conflict.  These  effects  might  include  how  many 
targets  an  entity  prosecutes,  how  quickly  a  entity  runs  out  of  ammunition,  or  even 
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the  survivability  of  the  entity  within  the  simulation.  To  achieve  realism,  the  model 
must  have  adequate  exit  criteria. 

One  technique  would  be  to  select  a  BDA  goal  (e.g.  reach  K-kill)  at  the  beginning 
of  each  engagement.  A  random  number  would  be  drawn  after  each  shot  (with  update) 
and  the  kill  state  would  be  determined  by  the  a/<'"')  vector.  If  the  selection  of  Xn 
meets  or  exceeds  the  BDA  goal  the  model  exits  the  BDA  process.  This  allows  for 
incorrect  estimation  of  a  target’s  kill  state  dne  to  the  uncertainty  on  the  battlefield. 
For  instance,  in  an  engagement  sequence,  the  tank  gunner  may  determine  BDA  as 
an  MF-kill  after  the  first  shot  when,  in  fact,  ground  truth  is  an  M-kill  or  perceive  an 
M-kill  when  ground  truth  is  K-kill. 

Another  technique  would  involve  having  a  goal  corresponding  to  threshold  of 
density  at  or  above  a  kill  state  (e.g.  at  least  90%  in  the  M-,  MF-,  and  K-kill  states). 
The  gunner  would  fire  until  this  goal  was  reached  and  the  target  would  be  presumed 
as  at  least  an  M-kill.  In  the  example  engagement  sequence  presented  in  Section  4.3.2, 
the  gunner  would  stop  after  the  third  shot  because  >  0.98  of  the  density  in  a/(3l  is 
contained  in  M-,  MF-,  and  K-kill  states.  This  would  likely  result  in  over-killing  the 
target,  or  shooting  more  rounds  than  necessary  in  many  engagements.  This  result 
actually  captures  reality  quite  well,  as  soldiers  are  more  apt  to  underestimate  BDA 
due  to  the  (survivability)  risks  involved. 

Yet  another  technique  would  create  some  function  to  incorporate  the  tradeoffs 
between  achieving  a  BDA  goal  on  the  current  target  and  engaging  another  (possibly 
more  important)  target.  If  some  critical  value  was  not  met  during  the  engagement 
the  entity  would  exit  the  process  and  engage  a  more  critical  target.  As  an  example, 
suppose,  in  an  engagement  sequence,  that  the  tank  gunner  perceives  that  the  target  is 
M-killed  after  the  first  shot  but  would  like  to  achieve  a  K-kill.  However,  a  new  enemy 
poses  an  immediate  and  more  serious  threat  to  the  tank.  The  model  might  decide  to 
move  to  the  higher  priority  target  based  on  this  information. 
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The  technique  chosen  to  provide  exit  criteria  for  an  engagement  will  certainly 
shape  the  implementation  of  the  methodology  into  a  combat  simulation  model.  As 
a  result,  this  choice  is  of  utmost  importance  to  the  decision  maker  and  should  be 
considered  carefully. 

5.3  Model  Assumptions  and  Strengths 

The  methodology  relies  on  several  assumptions  that  allow  a  mathematically 
valid  model  of  the  BDA  process.  The  main  assumptions  that  underly  the  DTMC 
with  updating  approach  are  as  follows: 

1.  The  event  sets  contain  mutually  exclusive  events. 

2.  The  probability  of  a  situation,  or  intersection  of  events  can  be  calculated. 

3.  The  probability  of  the  next  state  depends  only  on  the  current  state.  That  is, 
the  Markovian  property. 

4.  The  conditional  probabilities  dependent  on  the  current  state  are  time  homoge¬ 
neous.  That  is,  the  stationary  property. 

The  strengths  of  this  modeling  technique  are  extensive.  Specifically,  it  addresses 
several  of  the  shortfalls  of  the  current  AMSAA  heuristic  (reference  Section  1.3).  An 
item  already  discussed  is  incorporating  uncertainty  into  the  model.  Allowing  the 
target  to  jump  between  any  two  perceived  states  and  having  the  assessing  soldier 
perceive  incorrect  BDA  is  an  important  aspect  of  the  model. 

Another  issue  the  BDA  model  resolves  is  the  incorporation  of  situational  factors 
into  an  assessment.  The  model  directly  makes  use  of  the  perception  information 
available  to  a  soldier  performing  an  assessment,  providing  a  realistic  representation  of 
the  decision  process.  By  using  information  contained  in  the  situational  parameters, 
meaningful  probabilities  are  assigned  to  assessing  a  target  at  a  kill  state. 

Though  not  specifically  addressed  in  this  research,  dealing  with  indirect  fire 
could  be  easily  accomplished  with  this  approach.  Modeling  the  process  as  a  DTMC 
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provides  avenues  to  deal  with  missing  data,  a  likely  situation  for  indirect  fire.  Though 
the  model  has  numerous  strengths  and  advantages  over  the  previous  AMSAA  heuris¬ 
tic,  it  still  holds  many  avenues  for  improvement  and  further  study. 

5.4  Recommendations  for  future  research 

This  thread  of  research  provides  numerous  continuation  and  related  topics  that 
are  of  interest.  Since  this  marks  the  first  BDA  model  of  its  kind,  surely  improvement 
opportunities  are  available.  The  most  immediate  value-added  would  come  from  adding 
a  module  that  captures  the  degradation  of  BDA  capability  due  to  hindering  factors. 
These  factors  might  include  weather,  time  of  day,  range  to  the  target,  the  sensor  being 
used,  or  obscurants  (e.g.  dust  and  smoke).  Clearly  any  of  these  may  affect  the  quality 
of  the  perceived  information. 

Another  improvement  would  involve  an  exploration  of  the  dependence  relation¬ 
ships  among  event  sets.  Through  collection  of  real  data  from  combat  SMEs,  the 
assumptions  made  in  the  methodology  could  be  challenged  and  modified  if  necessary. 
This  would  likely  occur  in  conjunction  with  development  of  decision  tables  for  use  in 
a  combat  model. 

Additionally,  this  research  has  dealt  entirely  within  the  construct  of  Army  force- 
on-force  combat  models.  The  incorporation  of  uncertainty  perceived  information 
translates  well  to  the  BDA  process  in  any  realm  (air,  sea,  or  land).  As  a  result, 
this  methodology  could  prove  useful  in  other  DoD  simulations  with  similar  decision 
processes  and  should  be  investigated. 

Lastly,  any  number  of  improvement  techniques  might  be  applied  to  this  method¬ 
ology.  The  threads  above  represent  just  a  fraction  of  the  possible  topics  that  can  be 
reached  with  this  research. 
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Appendix  A.  SME  Survey  Design 

The  following  pages  present  the  proposed  data  collection  method.  SMEs  will  be 
determined  by  the  Army  per  their  rules  and  regulations. 
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Battle  Damage  Assessment  Questionnaire 


Position  Code: _  Position  Name: _ 

Years  Experience: _ 

Have  you  ever  perfonned  real  time  BDA  in  a  combat  situation?  Yes  /  No 

If  yes,  on  which  target  types  have  you  perfonned  BDA?  (Please  check  all  that  apply,  adding  any 
not  listed.) 


Tank 

APC 

Truck 

Infantry  Soldier 


Artillery  Piece 

Instructions 

For  each  question  you  will  be  presented  a  scenario  and  be  asked  to  identify  key  things  that 
you  might  see  in  that  situation.  There  will  be  three  sets  of  check  boxes  beside  each  item  labeled 
“N”  “M”  and  “A”.  Items  that  you  would  never  or  rarefy  expect  to  see  in  the  situation  should  be 
checked  in  the  “N”  column.  Check  items  you  might  see  in  the  “M”  column  and  finally  check 
critical  items  that  you  would  always  expect  to  see  in  the  “A”  column.  You  may  also  use  the 
space  provided  to  explain  your  thought  process  or  add  items  not  listed. 

You  should  assume  that  you  are  at  ID  level  using  the  most  capable  sensor  available  to 
you.  Try  to  be  as  complete  as  possible. 


The  following  are  definitions  of  battle  damage  assessment  (BDA)  kill  types  that  you  may  be 
asked  to  describe.  These  definitions  applicable  to  vehicles  but  the  concepts  extend  to 
dismounted  troops  as  well. 

M-kill  (Mobility):  A  target  is  subject  to  an  M-kill  if  it  is  incapable  of  executing  controlled 
movement  and  the  damage  is  not  repairable  by  the  crew  on  the  battlefield.  Failure  to  function 
may  be  caused  by  the  incapacitation  of  the  crew  or  damage  to  propulsion  or  control  equipment. 

F-kill  (Firepower):  A  target  is  subject  to  an  F-kill  if  it  is  incapable  of  delivering  controlled  fire 
from  the  main  armament  and  the  damage  is  not  repairable  by  the  crew  on  the  battlefield.  The  loss 
of  this  function  may  be  caused  by  the  incapacitation  of  the  crew  or  damage  to  the  main  armament 
and  its  associated  equipment. 

K-kill  (Catastrophic):  A  target  is  subject  to  a  K-kill  if  it  sustains  both  an  M-  and  F-kill  and  is 
damaged  to  the  extent  that  is  not  economically  repairable.  A  K-kill  is  more  likely  to  be  apparent 
to  the  crew  of  a  weapon  system  because  of  the  resulting  fires/detonation  of  ammunition. 


1)  An  enemy  artillery  piece  has  taken  fire  and  it  sustains  an  F-kill.  Please  mark  the 
appropriate  box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the 
situation.  If  it  is  true  that  the  artillery  piece  is  an  F-kill,  check  items  you  would  never  or 
rarely  see  in  the  “N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you 
would  always  see  in  the  “A”  column. 

Below  are  some  pictures  of  enemy  artillery  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment. 


Actions 

•  Engaging  targets 

o  Yes 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Piece 
o  Other/No  Action 


N  M  A 

□  □□ 


Physical  Damage 

•  Gun  Barrel 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Chassis/Platform 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


N  M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


2)  An  enemy  APC  has  taken  fire  but  it  does  not  sustain  a  kill.  Please  mark  the  appropriate 
box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is 
true  that  the  APC  has  not  sustained  a  kill,  check  items  you  would  never  or  rarely  see  in 
the  “N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you  would  always 
see  in  the  “A”  column. 

Below  are  some  pictures  of  enemy  APCs  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □  □  □ 

•  Activity 

o  Seeking  Cover  __ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  APC 
o  Other/No  Action 


Physical  Damage  N  M  A 

•  Weapons  Systems 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Hull 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Tracks/Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


3)  An  enemy  truck  has  taken  fire  and  it  sustains  an  M-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  truck  is  an  M-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  trucks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  I 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □□□ 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Truck 
o  Other/No  Action 


Physical  Damage 

•  Body 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


4)  An  enemy  APC  has  taken  fire  and  it  sustains  a  K-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  APC  is  a  K-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  APCs  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □  □  □ 

•  Activity 

o  Seeking  Cover  __ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  APC 
o  Other/No  Action 


Physical  Damage  N  M  A 

•  Weapons  Systems 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Hull 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Tracks/Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


5)  An  enemy  artillery  piece  has  taken  fire  and  it  sustains  a  K-kill.  Please  mark  the 
appropriate  box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the 
situation.  If  it  is  true  that  the  artillery  piece  is  a  K-kill,  check  items  you  would  never  or 
rarely  see  in  the  “N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you 
would  always  see  in  the  “A”  column. 

Below  are  some  pictures  of  enemy  artillery  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment. 


Actions  N  M  A 

•  Engaging  targets 

o  Yes  □□□ 

•  Activity 

o  Seeking  Cover  _ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Piece 
o  Other/No  Action 


Physical  Damage 

•  Gun  Barrel 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Chassis/Platform 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


N  M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


6)  An  enemy  tank  has  taken  fire  and  it  sustains  a  K-kill.  Please  mark  the  appropriate  box  for 
each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  tank  is  a  K-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  tanks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engagement 

o  Engaging  Targets  □  □□ 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Tank 
o  Other/No  Action 


Physical  Damage 


•  Turret 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Hull 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Tracks 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


7)  An  enemy  truck  has  taken  fire  and  it  sustains  a  K-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  truck  is  a  K-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  trucks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  I 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □□□ 

•  Activity 

o  Seeking  Cover  _ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Truck 
o  Other/No  Action 


Physical  Damage 

•  Body 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


8)  An  enemy  APC  has  taken  fire  and  it  sustains  an  F-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  APC  is  an  F-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  APCs  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □  □  □ 

•  Activity 

o  Seeking  Cover  __ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  APC 
o  Other/No  Action 


Physical  Damage  N  M  A 

•  Weapons  Systems 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Hull 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Tracks/Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


9)  An  enemy  tank  has  taken  fire  but  it  has  not  sustained  a  kill.  Please  mark  the  appropriate 
box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is 
true  that  the  tank  did  not  sustain  a  kill,  check  items  you  would  never  or  rarely  see  in  the 
“N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see 
in  the  “A”  column. 

Below  are  some  pictures  of  enemy  tanks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engagement 

o  Engaging  Targets  □  □□ 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Tank 
o  Other/No  Action 


Physical  Damage 


•  Turret 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Hull 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Tracks 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


10)  An  enemy  truck  has  taken  fire  but  does  not  sustain  a  kill.  Please  mark  the  appropriate 
box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is 
true  that  the  truck  has  not  sustained  a  kill,  check  items  you  would  never  or  rarely  see  in 
the  “N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you  would  always 
see  in  the  “A”  column. 

Below  are  some  pictures  of  enemy  trucks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  I 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □□□ 

•  Activity 

o  Seeking  Cover  _ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Truck 
o  Other/No  Action 


Physical  Damage 

•  Body 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


11)  An  enemy  APC  has  taken  fire  and  it  sustains  an  M-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  APC  is  an  M-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  APCs  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □  □  □ 

•  Activity 

o  Seeking  Cover  __ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  APC 
o  Other/No  Action 


Physical  Damage  N  M  A 

•  Weapons  Systems 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Hull 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Tracks/Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


12)  An  enemy  tank  has  taken  fire  and  it  sustains  an  M-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  tank  is  an  M-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  tanks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engagement 

o  Engaging  Targets  □  □□ 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Tank 
o  Other/No  Action 


Physical  Damage 


•  Turret 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Hull 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

•  Tracks 

o 

No  Damage 

o 

Light  Damage 

o 

Fleavy  Damage 

Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


13)  An  enemy  artillery  piece  has  taken  fire  but  does  not  sustain  a  kill.  Please  mark  the 
appropriate  box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the 
situation.  If  it  is  true  that  the  artillery  piece  did  not  sustain  a  kill,  check  items  you  would 
never  or  rarely  see  in  the  “N”  column,  items  you  might  see  under  the  “M”  column,  and 
items  you  would  always  see  in  the  “A”  column. 

Below  are  some  pictures  of  enemy  artillery  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment. 


Actions 

•  Engaging  targets 

o  Yes 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Piece 
o  Other/No  Action 


N  M  A 

□  □□ 


Physical  Damage 

•  Gun  Barrel 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Chassis/Platform 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


N  M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


14)  An  enemy  tank  has  taken  fire  and  it  sustains  an  F-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  tank  is  an  F-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  tanks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  _  _ 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engagement 

o  Engaging  Targets  □  □□ 

•  Activity 

o  Seeking  Cover  _  __ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Tank 
o  Other/No  Action 


Physical  Damage 


•  Turret 

o 

No  Damage 

o 

Light  Damage 

o 

Heavy  Damage 

•  Hull 

o 

No  Damage 

o 

Light  Damage 

o 

Heavy  Damage 

•  Tracks 

o 

No  Damage 

o 

Light  Damage 

o 

Heavy  Damage 

N  M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


15)  An  enemy  truck  has  taken  fire  and  it  sustains  an  F-kill.  Please  mark  the  appropriate  box 
for  each  of  the  actions  or  characteristics  that  you  might  see  given  the  situation.  If  it  is  true 
that  the  truck  is  an  F-kill,  check  items  you  would  never  or  rarely  see  in  the  “N”  column, 
items  you  might  see  under  the  “M”  column,  and  items  you  would  always  see  in  the  “A” 
column. 

Below  are  some  pictures  of  enemy  trucks  that  you  may  use  to  mark  or  illustrate  your 
expectations  of  the  assessment 


Actions  N  M  A 

•  Movement 

o  Toward  your  position  I 

o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes  □□□ 

•  Activity 

o  Seeking  Cover  _ 

o  Taking  a  Firing  Position 
o  Personnel  Abandoning  the  Truck 
o  Other/No  Action 


Physical  Damage 

•  Body 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 

•  Wheels 

o  No  Damage 
o  Light  Damage 
o  Heavy  Damage 


M  A 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


For  soldiers  it  does  not  make  sense  to  use  the  M-,  F-  and  K-kill  types  described  in  the  directions 
at  the  beginning  of  this  survey.  Instead  we  will  classify  an  enemy  soldier  as  Incapacitated  if  he 
cannot  continue  the  mission,  assault,  or  defend.  Incapacitation  may  include  fatality  but  does  not 
necessarily  imply  that  the  enemy  is  dead. 

16)  An  enemy  soldier  has  taken  fire  and  he  sustains  an  incapacitation-kill.  Please  mark  the 
appropriate  box  for  each  of  the  actions  or  characteristics  that  you  might  see  given  the 
situation.  If  it  is  true  that  the  soldier  is  incapacitated,  check  items  you  would  never  or 
rarefy  see  in  the  “N”  column,  items  you  might  see  under  the  “M”  column,  and  items  you 
would  always  see  in  the  “A”  column. 


Actions 

•  Movement 

o  Toward  your  position 
o  Away  from  your  position 
o  Parallel  to  your  position 
o  None 

•  Engaging  targets 

o  Yes 

•  Activity 

o  Seeking  Cover 
o  Taking  a  Firing  Position 
o  Falling  to  the  Ground 
o  Other/No  Action 


N  M  A 


□  □□ 


Physical  Damage 


Head 

o 

Wounded 

o 

Not  Wounded 

Torso 

o 

Wounded 

o 

Not  Wounded 

Limbs 

o 

Wounded  Arms 

o 

Wounded  Legs 

o 

No  Wounds 

N  M  A 

BBS 

BBB 


Please  use  the  space  below  to  comment  on  your  choices  and/or  add  things  that  are  not  listed. 


Degradation  Questions 


Now  consider  making  assessments  with  different  sensors.  Most  likely  each  sensor  is  sensitive  to 
different  factors  that  will  degrade  your  ability  to  see  the  target  and  make  an  assessment.  You 
will  be  asked  to  mark  how  much  each  factor  degrades  the  given  sensor. 

Key: 

N:  No  Degradation 
H:  Half  Degradation 
F :  Full  Degradation 

Comments:  Use  the  space  given  to  describe  how  the  factor  affects  the  sensor. 

Complete  the  following  tables  concerning  different  sensor  types.  For  example  if  your  sensor  is 
unaided  visual,  ambient  light  may  have  a  full  degradation  effect.  Comments  would  include  that 
nighttime  (low/no  ambient  light)  impairs  your  ability  to  see.  If  there  are  additional  factors  that  do 
not  appear  on  the  list,  add  them  and  answer  in  the  same  manner. 


IR/Thermal 

Factor  N  H  F  Comments 


Ambient  Light 

Humidity 

Obscurants 

Range  to  Target 

Weather 

Movement  while  Assessing 

Unaided  Visual 

Factor  N  H  F  Comments 


Ambient  Light 

Humidity 

Obscurants 

Range  to  Target 

Weather 

Movement  while  Assessing 

Appendix  B.  MATLAB  Code  for  Simulation  of  Survey  Responses 

1 

function  []  =  MasterSimulationO 

2 

°/0  MasterSimulation:  a  function  to  simulate  survey  responses,  calculate 

3 

'/.  estimated  and  true  a’ (n)  vectors,  record  data,  and 

4 

°/0  create  plots 

7 

6 

/  Calls:  ReadSimulationProbabilities ,  ReadTrueProbabilities , 

7 

°/0  SurveyResponseMatrix,  PlotDataConvergence ,  DTMC,  GetSituation, 

8 

/  DataCalcuations ,  GetStats,  N_Plots,  Engage Sequence 

°/o 

10 

^^c:|e3|e^C3|e^C3f;3|e^C3f:3|e^c3|c3|e^c3|e^e4:3le:lc:te3le:lc,li::ic:4C3le3ie^cHc3ieHe3le3le3f::iC3lC3f:3le:4C3lc3ieHi:3le3ic,le3le:le3ii:3le:4c:f:3iC3lC3k^^He:lc,li:3k3|c3f:3ie:4c3f:3fe3lcHc3ie^He:ic:f: 

11 

/  Author:  Michael  Carras 

12 

°/0  michael .  carrasOaf  it .  edu 

13 

i - 

14 

'/.  Created:  2006 

15 

/  Last  Modified:  9  March  2006 

16 

17 

7. 

18 

7,  VARIABLES  (Global/Main) - 

19 

l  Pio_PhysicalDamage  -  Simulation  probabilities  for  binary  response 

20 

l  vector  of  physical  damage 

21 

°/0  CondPio_Actions  -  Conditional  simulation  probabilities  for  binary 

22 

°/0  response  vector  of  actions  given  phys  damage 

23 

°/0  Ptrue_All  -  True  marginal  event  probabilities 

24 

'/.  CondPtrue_Actions  -  True  conditional  ‘  ‘Action’  *  Probabilities 

25 

/  NumResponse  -  Number  of  survey  responses 

26 

°/0  ResponseMatrix  -  Matrix  containing  all  survey  responses 

27 

/  aPnTrue  -  True  a’  vector 

28 

'/.  a_nTrue  -  True  a  vector 

29 

/  Situation  -  Matrix  of  situations  tested 

30 

l  sit  -  Current  situation  counter 

31 

'/.  B_n  -  Current  siutation 

32 

°/0  Bn_pd  -  Pointer  to  correct  row  of  conditional  action 

33 

'/.  probabilities 

34 

/  P_BnaTrue  -  True  conditional  probabilities  for  current 

35 

°/0  situation 

36 

°/0  P_BnTrue  -  True  marginal  probabilities  for  current  situation 

37 

/  temp  -  temporary  vector  holding  intersection  probs 

38 

'/.  tempSum  -  sum  of  temp 

39 

/  PlotData  -  Matrix  with  data  for  plots 

40 

°/0  Mean  -  Mean  BDA  based  on  a’ 

41 

°/0  Var  -  Variance  of  BDA  based  on  a’ 

42 

l  SD  -  Standard  Deviation  of  BDA  based  on  a’ 

43 

'/.  CV  -  Coefficint  of  variation  based  on  a’ 

44 

i - 

45 

46 

47 

48 

49 

50 

51 

52 

53 

54 

55 

56 

57 

58 

59 

60 
61 
62 

63 

64 

65 

66 

67 

68 

69 

70 

71 

72 

73 

74 

75 

76 

77 

78 

79 

80 
81 
82 

83 

84 

85 

86 

87 

88 


7,  Closes  all  open  plots  and  turns  warning  displays  off 
warning  off  all 
close  all 

/  Read  simulation  Probabilities  and  True  Probabilities 
[Pio_PhysicalDamage ,CondPio_Actions]  =  ReadSimulationProbabilities () ; 
[Ptrue_All ,CondPtrue_Actions]  =  ReadTrueProbabilities () ; 

7,  Set  the  number  of  responses  here 
NumResponse  =  1000; 

7,  generate  survey  responses 

[ResponseMatrix]  =  SurveyResponseMatrix (NumResponse , . . . 
Pio_PhysicalDamage , CondPio_Actions , 0) ; 

7,  Plot  convergence  of  data 

PlotDataConvergence (ResponseMatrix , NumResponse) ; 

7,  initialize  variables 
aPnTrue  =  [10000]; 
a_nTrue  =  DTMC ( aPnTrue ) ; 

7,  retrieve  B_n’s 

[Situation]  =  GetSituations () ; 

7,  create  data  for  plots 
for  sit  =  1:5 

B_n  =  Situation(sit , : ) ; 

Bn_pd  =  l+(B_n(4) -9) +3* (B_n(5) -12)+9* (B_n(6) -15) ; 

P_BnaTrue  =  squeeze ( [CondPt rue_ Act ions (Bn_pd,B_n(l) ,:),... 
CondPtrue_Actions (Bn_pd,B_n(2) ,:),... 

CondPtrue_Actions (Bn_pd,B_n(3) ,:)]); 

P_BnTrue  =  [P_BnaTrue ;  ... 

Ptrue_All ( : ,B_n(4) ) ’ ;  ... 

Ptrue_All ( : ,B_n(5)) > ; . . . 

Ptrue_All ( : ,B_n(6)) J] ; 

temp  =  prod(P_BnTrue , 1) . *a_nTrue ; 
tempSum  =  sum (temp) ; 
aPnTrue  =  temp /tempSum; 
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89 

90  [PlotData]  =  DataCalculations (NumResponse ,B_n,ResponseMatrix, aPnTrue) ; 

91  [Mean, Var ,SD,CV]  =  GetStats (aPnTrue) ; 

92  PlotData  =  [PlotData; [1, aPnTrue, Mean, Var, SD,CV,0]]  ; 

93  xlswrite( ’Ch4PlotData.xls ’ , PlotData, sit , ,b2*) ; 

94 

95  N_Plots (PlotData, sit , NumResponse) ; 

96 

97  end 

98 

99  EngageSequence (ResponseMatrix , Ptrue_All , CondPtrue_Actions) 

100  end 

101  ^^3|ofe3f:He:le3(e3le:lc,f:3fe3|cH‘9fe:4eHC3i(^He:lc:te3le:lC3fi:9ic:4e3le^:t:Hole^He3le3(e:ie:t:3f:3lc:4C3l0fe^He:(C3f:3le:lC3f:3le:t:3f::ie:4e3lo|eHe3le:4C3ii:3le3lc:(C9fe:4c3(e3fe:4cHoie^HC3le^ 

102 

103 

104  l  Sub  functions 

105  ^$Nc^$He:^3le3le:le3f:3lc:4C3lc:ie:4cNc3lc’li:3le3lc:le3le:lc,t::ic:lC3lc>le:4cNc:ie$He:lc:ii::ic:4C3f:3ic:4C3lc3ieHe3le3le3lC3le:lC3t:3lc:4c:t::ic:4C3lc$He3le:le3ii:3lc3lc3t:3ie3lc3le3le:4cNc3ie3|cNC3ie’t: 

106  function [Pio_PhysicalDamage ,CondPio_Actions] =ReadSimulationProbabilities () 

107  °/0  ReadSimulationProbabilities :  a  function  to  read  in  the  probabilities  of 

108  7,  simulated  surevey  responses  from  an  excel  spreadsheet 

109 

110  Pio_PhysicalDamage  =  xlsread(’ SurveyProbs.xls’ , ’p_0’ , ’b2: j6’) ; 

111 

112  CondPio_Actions ( : , : , 1)  =  xlsread( ’SurveyProbs . xls ’ , ’p_0 J , ’bl8 : i44’ ) ; 

113  CondPio_Actions ( : , : ,2)  =  xlsread( ’SurveyProbs .xls *  , ,p_0> , ’b46 : i72’ ) ; 

114  CondPio_Actions ( : , : ,3)  =  xlsread( ’SurveyProbs . xls ’ , ’p_0 J , ’b74 : ilOO 1 ) ; 

115  CondPio_Actions ( : , : ,4)  =  xlsread( ’SurveyProbs . xls ’ , ’p_0 * , ’bl02 : il28’ ) ; 

116  CondPio_Actions( : , : ,5)  =  xlsread( ’SurveyProbs . xls ’ , ’p_0 } , ’bl30 : il56 * ) ; 

117 

118  end 

119 

120 

121  ^$He$$He^Hi:He$3t:3lc:lC3lc3ie3lcHe:le,li:3le:lc:le3le:lc,li::ic:lC3le3le=4cNc:ieHe3le3le3te:iC3lC3f:9le=4cNe3ie$He3lc,li:3le:lC3t:3le:4c:t::iC3lC3le:ie,lC3le:lc,t:3lc3lc’t:3ie3lc3t:3le:lcNc:ie3f:’k:icHi: 

122  function  [Ptrue_All ,CondPtrue_Actions]  =  ReadTrueProbabilities () ; 

123  /  ReadSimulationProbabilities:  a  function  to  read  in  true  probabilities  of 

124  7,  events  from  and  excel  spreadsheet 

125 

126  Ptrue_All  =  xlsread( ’SurveyProbs .xls’ , ’p_0’ , ’bllrrlS’) ; 

127 

128  CondPtrue_Actions ( : , : , 1)  =  xlsread( ’SurveyProbs .xls ’ , ’p.O’ , Jkl8 :r44’ ) ; 

129  CondPtrue_Actions( : , : ,2)  =  xlsread( ’ SurveyProbs .xls ’ , Jp_0’ , Jk46 :r72’ ) ; 

130  CondPtrue_Actions( : , : ,3)  =  xlsread( ’SurveyProbs .xls ’, Jp_0’ , ,k74:rl00’ ) ; 

131  CondPtrue_Actions ( : , : ,4)  =  xlsread( ’SurveyProbs .xls ’ , Jp_0’ , Jkl02 :rl28’ ) ; 
CondPtrue_Actions ( : , : ,5)  =  xlsread( ’ SurveyProbs . xls ’ , ’p_0’ , Jkl30:rl56J) ; 
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132 


133 

134 

135 

136 

137 

138 

139 

140 

141 

142 

143 

144 

145 

146 

147 

148 

149 

150 

151 

152 

153 

154 

155 

156 

157 

158 

159 

160 
161 
162 

163 

164 

165 

166 

167 

168 

169 

170 

171 

172 

173 

174 

175 

176 


end 


3|e3|c3fe3fc3|e3|e3|c3|c3|e3(e3|c3|c3|e3|c3(c3|e3fc3(c3|e3|e3(e3|e3|e3tc3)c3(e3|e3fc3|c3(e3fc3(e3|e3|e3)c3)c3|c3tc3fc3|e3(e3(e3te3|e3fe3te3|e3|c3te3fc3|e3tc3fc3|e3|e3fc3tc3|e3|c3(e3|e3|c3fc34c3|e3fc3)c3(c3|e3fe3tc3|e3(e3|c 

function  [ResponseMatrix]  =  SurveyResponseMatrix (NumResponse ,.. . 

Pio_PhysicalDamage ,CondPio_Actions ,Ex) 

°/0  SurveyResponseMatrix:  a  function  to  simulate  (NumResponse)  number  of 

°/0  survey  responses  by  rendom  numer  draws 

#/. 

l  Calls:  DamageCondition,  ConvertActions 

y. 

/  LOCAL  VARIABLES 
’/.  i 

l  iResponseMatrix 
i„  Rnd_io_pd 
#/.  Rnd_io_a 
'/•  n 

°/0  etaRnd_io_pd 
/  etaRnd_io_a 
#/.  TempPio_pd 
#/,  Rsp_io_pd 
°/0  Rsp_num_pd 
°/0  Weight_pd 

°/o 

y,  TempPio_a 
°/0  Rsp_io_a 
0/.  Rsp_num_a 
°/0  etaRsp_num 
/ - 

°/0  Initialize  ResponseMatrix 
ResponseMatrix  =  zeros (NumResponse, 17,5) ; 

i  Iterate  thorough  kill  states 
for  i  =  1:5 

1  initialize  the  response  matrix  for  this  kill  state  and  generate 
%  random  numbers 

iResponseMatrix  =  zeros (NumResponse, 17) ; 

Rnd_io_pd  =  rand (NumResponse , 9) ; 

Rnd_io_a  =  rand (NumResponse ,8) ; 

for  n  =  1 : NumResponse 

i  Get  the  random  numbers  for  this  reponse 


-  kill  state  counter 

-  placeholder  for  ith  layer  of  ResponseMatrix 

-  matrix  of  random  numbers  for  physical  damage 

-  matrix  of  random  numbers  for  actions 

-  Number  of  responses  counter 

-  nth  row  of  Rnd_io_pd 

-  nth  row  of  Rnd_io_a 

-  appropriate  row  of  simulation  probabilities 

-  binary  vector  of  physical  damage  responses 

-  physical  damage  response  in  numerical  form 

-  Weight  vector  for  actions  to  condition  on 

physical  damage 

-  appropriate  simulation  probabilities  for  actions 

-  binary  vector  of  action  responses 

-  action  response  in  numerical  distribution  form 

-  concatenation  of  action  and  pysical  damage 


177 


178 

179 

180 
181 
182 

183 

184 

185 

186 

187 

188 

189 

190 

191 

192 

193 

194 

195 

196 

197 

198 

199 

200 
201 
202 

203 

204 

205 

206 

207 

208 

209 

210 
211 
212 

213 

214 

215 

216 

217 

218 

219 

220 


etaRnd_io_pd  =  Rnd_io_pd(n, : ) ; 
etaRnd_io_a  =  Rnd_io_a(n, : ) ; 

l  Get  the  appropriate  kill  state  physical  damage  event 
/  probabilities  and  compare  against  random  numbers 
TempPio_pd  =  Pio_PhysicalDamage (i , : ) ; 

Rsp_io_pd  =  etaRnd_io_pd  <=  TempPio_pd; 

l  convert  the  binary  physical  damage  to  numerical  probability 
7#  distributions  and  get  the  weight  vector  for  action  responses 
[Rsp_num_pd,Weight_pd]  =  DamageCondition(Rsp_io_pd, i) ; 

l  Get  appropriate  kill  state  conditional  probabilities  and  ocmpare 

l  them  against  random  numbers 

TempPio_a  =  squeeze (CondPio_Actions( : , : ,i)) ; 

TempPio_a  =  Weight_pd*TempPio_a; 

Rsp_io_a  =  etaRnd_io_a  <=  TempPio_a; 

i  convert  the  binary  actions  to  numerical  probability 
%  distributions 

[Rsp_num_a]  =  ConvertActions (Rsp_io_a, i) ; 

l  place  this  reponse  in  the  iResponseMatrix 
etaRsp_num  =  [Rsp_num_a,  Rsp_num_pd] ; 
iResponseMatrix (n, : )  =  etaRsp_num; 

end 

“/*  place  the  ith  layer  into  ResponseMatrix 
ResponseMatrix( : , : , i)  =  iResponseMatrix; 

7*  write  data  to  spreadsheet 
if  Ex  ==  0 

xlswrit e ( ’ Ch4SimData . xls  * , iResponseMatrix , i , ’ b2 5 ) ; 
elseif  Ex  ==  1 

xlswrite ( ’ Ch4ExSimData . xls  ’ , iResponseMatrix , i , *b2  * ) ; 

end 

end 

end 

7o\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ 

function  [Rsp_num_pd, Weight _pd]  =  DamageCondition(Rsp_io_pd, i) ; 

7,  DamageCondition:  function  to  convert  binary  rsponses  to  numerical  ones 
7.  and  produce  a  conditioning  weight  vector  for 
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222 
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226 

227 

228 

229 

230 

231 

232 

233 

234 

235 

236 

237 

238 

239 

240 

241 

242 

243 

244 

245 

246 

247 

248 

249 

250 

251 

252 

253 

254 

255 

256 

257 

258 

259 

260 
261 
262 

263 

264 


l  actions 

7. 

7,  LOCAL  VARIABLES - 

7,  tur_io,  hul_io,  trk_io  -  binary  response  pieces  for  turret,  hull,  tracks 
7,  tur_num,  hul_num,  trk_num  -  numerical  responses  for  turret,  hull,  tracks 

7. - 


7,  break  logical  vectors  into  turret,  hull,  track  pieces 
tur_io  =  [Rsp_io_pd(l) ,Rsp_io_pd(2) ,Rsp_io_pd(3)] ; 
hul_io  =  [Rsp_io_pd(4) ,Rsp_io_pd(5) ,Rsp_io_pd(6)] ; 
trk_io  =  [Rsp_io_pd(7) ,Rsp_io_pd(8) ,Rsp_io_pd(9)] ; 
tur_num  =  []  ;  hul_num  =  []  ;  trk_num  =  []  ; 

7,  assign  distribution  to  turret 
tur_num ( ~tur_io) =  0; 
switch  sum(tur_io) 
case(0) 

switch  i 
case(l) 

if  rand  <  .75  tur_num  =  [1,0,0]; 
else  tur_num  =  [.5,. 5,0];  end 
case(2) 

if  rand  <  .75  tur_num  =  [0,.5,.5]; 
else  tur_num  =  [1/3, 1/3, 1/3] ;  end 
case{3,4,5} 

if  rand  <  .75  tur_num  =  [0,0,1]; 
else  tur_num  =  [0,.5,.5];  end 

end 

case(l) 

tur_num(tur_io)  =  1; 
case(l) 

tur_num(tur_io)  =  1; 
case{2} 

tur_num(tur_io)  =  .5; 
case{3} 

tur_num(tur_io)  =  1/3; 

end 

7,  assign  distribution  to  hull 
hul_num(~hul_io) =  0; 
switch  sum(hul_io) 
case(0) 

switch  i 

case{l ,2,3} 
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281 
282 

283 

284 
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286 
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288 
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if  rand  <  .75  hul_num  =  [1,0,0]; 
else  hul_num  =  [.5,. 5,0];  end 
case (4) 

if  rand  <  .75  hul_num  =  [.5,. 5,0]; 
else  hul_num  =  [1/3, 1/3, 1/3] ;  end 
case(5) 

if  rand  <  .75  hul_num  =  [0,0,1]; 
else  hul_num  =  [0,.5,.5];  end 

end 

case(l) 

hul_num(hul_io)  =  1; 
case{2} 

hul_num(hul_io)  =  .5; 
case{3} 

hul_num(hul_io)  =  1/3; 

end 

"/assign  distribution  to  tracks 
trk_num (~trk_io) =  0; 
switch  sum(trk_io) 
case(0) 

switch  i 

case{l,3} 

if  rand  <  .75  trk_num  =  [1,0,0]; 
else  trk_num  =  [.5,. 5,0];  end 
case{2 ,4} 

if  rand  <  .75  trk_num  =  [0,0,1]; 
else  trk_num  =  [0,.5,.5];  end 
case(5) 

if  rand  <  .75  trk_num  =  [1/3 , 1/3 , 1/3] ; 
else  trk_num  =  [0,.5,.5];  end 

end 

case(l) 

trk_num(trk_io)  =  1; 
case{2} 

trk_num(trk_io)  =  .5; 
case{3} 

trk_num(trk_io)  =  1/3; 

end 

7,  assign  output  1  (d_eta,i) 

Rsp_num_pd  =  [tur_num,  hul_num,  trk_num] ; 

7,  assign  output  2  (weight  vector) 
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309 


Weight_pd  =  zeros (1,27); 
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330 

331 

332 
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for  k  =  1:3 

for  h  =  1:3 

for  t  =  1:3 

wt_pd  =  tur_num(t) *hul_num(h) *trk_imm(k) ; 

TempPos  =  t  +  3*(h-l)  +  9*(k-l); 

Weight_pd(TempPos)  =  wt_pd; 

end 

end 

end 

end 

#/.\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ 

function  [Rsp_num_a]  =  ConvertActions (Rsp_io_a, i) ; 

7.  ConverActions :  function  to  convert  binary  rsponses  to  numerical  ones 

7. 

/  LOCAL  VARIABLES  - 

l  mov_io,  eng_io,  act_io  -  binary  response  pieces  for  movement,  engagement, 
7.  activity 

7.  mov_num,  eng_num,  act_num  -  numerical  responses  for  movement,  engagement, 
7,  activity 

7. - 


7,  break  logical  vectors  into  movement,  engagement,  activity 
mov_io  =  [Rsp_io_a(l) ,Rsp_io_a(2) ] ; 
eng_io  =  [Rsp_io_a(3) ,Rsp_io_a(4)] ; 

act_io  =  [Rsp_io_a(5) ,Rsp_io_a(6) ,Rsp_io_a(7) ,Rsp_io_a(8)] ; 
mov_num  =  []  ;eng_num  =  []  ;  act_num  =  []  ; 


7,  assign  distribution  to  movementn 
mov_num(~mov_io)  =  0; 
switch  sum(mov_io) 
case(0) 

switch  i 

case{l,3} 

mov_num  =  [.5,. 5]; 
otherwise 

mov_num  =  [0,1]; 


end 

case(l) 

mov_num(mov_io)  =  1; 
case(2) 

mov_num(mov_io)  =  .5; 
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380 

381 
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end 

°/0  assign  distribution  to  engagement 
eng_num(~eng_io)  =  0; 
switch  sum(eng_io) 
case(0) 

switch  i 

case{l,2} 

eng_num  =  [ . 5 , . 5] ; 
otherwise 

eng_num  =  [0,1]; 

end 

case(l) 

eng_num(eng_io)  =  1; 
case(2) 

eng_num(eng_io)  =  .5; 

end 

°/0  assign  distribution  to  activity 
act_num(~act_io)=  0; 
switch  sum(act_io) 
case(0) 

switch  i 
case(l) 

if  rand  <  .75  act_num  =  [ . 25 , . 25 , . 25 , . 25] 
else  act_num  =  [1/3, 1/3,0, 1/3] ;  end 
case{2,4} 

if  rand  <  .75  act_num  =  [0,0,. 5,. 5]; 
else  act_num  =  [0,0, 1,0];  end 
case(3) 

if  rand  <  .75  act_num  =  [1/3,0, 1/3, 1/3] ; 
else  act_num  =  [.5,0, .5,0];  end 
case(5) 

act_num  =  [0,0, 0,1]; 

end 

case(l) 

act_num(act_io)  =  1; 
case{2} 

act_num(act_io)  =  .5; 
case{3} 

act_num(act_io)  =  1/3; 
case{4} 

act_num(act_io)  =  .25; 

end 
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397 


#/.  assign  output  (d_eta,i) 

Rsp_num_a  =  [mov_num,  eng_num,  act_num] ; 


398 

399 

400 

401 

402 
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405 

406 

407 

408 

409 

410 

411 

412 

413 

414 

415 

416 

417 

418 

419 

420 

421 

422 

423 

424 

425 

426 

427 

428 

429 

430 

431 

432 

433 

434 

435 

436 

437 

438 

439 

440 


end 


^^c:|e3|e3|o|e^c^C3|e^c4;3|e^:3|c3|e^:3|e^e^e3|c^c3|e3|e^c^c3|e^:3|e^e^:3|o|(^He:lc,ii:3le:t:3f:3lc:4C3le3le:te3le:(C3f:3le:lC3f:3le3lC3(e9ie:4e3loie^He:lC3ii:3le:lC3f::ic:4c,f:3fe:t:HC3le3(C3le:(e:lc 

function  []  =  PlotDataConvergence (ResponseMatrix,NumResponse) ; 
l  PlotDataConvergence:  function  to  plot  how  an  event  probability  converges 
/  to  its  { ‘true”  probability 

#/. 

/  LOCAL  VARIABLES  - 

/  P_Tnd  -  Probability  of  turret  with  no  damage  from 

°/0  ResponseMatrix 

/  pTrue_Tnd  -  True  probability  of  turret  with  no  damage 

y„ - 


y.  get  responses  for  turret  no  damage 
P_Tnd  =  squeeze (ResponseMatrix ( : ,9, 1) ) ; 

#/.  create  data  points  every  10  responses 
for  N  =  10: 10 : NumResponse 

d_Tnd(N/10)  =  mean(P_Tnd(l : N) ) ; 

end 

°/0  set  x  axis  support 
x  =  10: 10: NumResponse; 

pTrue_Tnd  =  ones (length (x) , 1) *0 . 66975 ; 

°/0  draw  figure  and  set  properties 
figure (1) 

plot (x,d_Tnd) 
hold  on 

plot (x , pTrue_Tnd , ’ -k ’ ) 
hold  off 
axl  =  gca; 

set (get (axl, ’XLabel’) , ’String’ , ’Number  of  Survey  Responses,  N’ ,  .  .  . 

’ Font Name ’, ’times  new  roman’); 
set (get (axl, ’YLabel’) , ’String’ , ’d_{ND}(  {\itT_{NoDamage}}  ) 

’ Font Name ’, ’times  new  roman’); 
set (axl , ’FontName ’, ’times  new  roman’,... 

’ YLim’ , [0  1]  ,  .  .  . 
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441 

’Xlim’ , [10  NumRe spons e] ) 

442 

grid  on 

443 

444 

end 

445 

446 

447 

^^c:|c^e3f;3|e^c^c3|e^c3f;3|e^c3|e^e^c3|e^e4;3|e^c^e3|c^c3|e^e3|c3|o|e^c3|c^e^e3|c^c^e^e^c3f:3|e^c3|c^e3|e3|e3|e3|o|e^e^e3|e^c^e^e^c3|e3|e^e3|c^c3f;3|c^c3f;^e3|c3f:3|e^c3|<:ie^He:ic,f: 

448 

function  [an]  =  DTMC(aPn) 

449 

'/.  DTMC:  function  to  multiply  aPn  and  the  transition  probability  matrix 

450 

451 

l  transition  probability  matrix,  P 

452 

P=[.2,  .2,  .2,  .2,  .2;... 

453 

0,  1/3,  0,  1/3,  1/3; .  .  . 

454 

0,  0,  1/3,  1/3,  1/3;... 

455 

0,  0,  0,  .5,  .5;... 

456 

o 

o 

o 

o 

457 

458 

an  =  aPn*P; 

459 

460 

end 

461 

462 

463 

^^C3|e^e3|e3|e^c^e3|e^c3f;3|e^c3|0|e^e3|c^e^0|e^c^c3|c^c^0|e^e3|c9|e^c3|0|e^e3|c^c^e^e^:3f;3|e^:3|0|e3|e3|e^e3f:3|e^e3f;3|e^:^c9|c^c3|e3|e^e3|e^c^e3|e^c^0|e^c3)e3|e^c3|e^e3f;3|e3|e3f; 

464 

function  [Situation]  =  GetSituations () 

465 

7,  GetSituations:  function  to  retrieve  the  correct  actions  and  physical 

466 

°/0  damage  for  plots 

467 

•/. 

468 

7,  LOCAL  VARIABLES - 

469 

7,  ObsAct  -  matrix  of  all  possible  action  combinations 

470 

7,  ObsPhysDam  -  matrix  of  all  possible  physical  damage  combos 

471 

•/. - 

472 

473 

ObsAct  =  [1,3,5;  .  .  .1 

474 

2,3,5; . . .2 

475 

1,4,5; . . .3 

476 

2,4,5; . . .4 

477 

1,3,6; . . .5 

478 

2,3,6; . . .6 

479 

1,4,6; . . .7 

480 

2,4,6; . . .8 

481 

1,3,7; . . .9 

482 

2,3,7; . .  .10 

483 

1,4,7; . . .11 

484 

2,4,7; . . .12 
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485 


1,3,8; . . .13 

486  2, 3, 8;... 14 

487  1 ,4,8; ...  15 

488  2,4,8;]  ;0/ol6 


489 

ObsPhysDam  - 

[9,12,15; . . .1 

490 

10,12,15; 

.  .  .2 

491 

11,12,15; 

.  .  .3 

492 

9,13,15; . 

.  .4 

493 

10,13,15; 

.  .  .5 

494 

11,13,15; 

.  .  .6 

495 

9,14,15; . 

.  .7 

496 

10,14,15; 

.  .  .8 

497 

11,14,15; 

.  .  .9 

498 

9,12,16; . 

.  .10 

499 

10,12,16; 

.  .  .11 

500 

11,12,16; 

.  .  .12 

501 

9,13,16; . 

.  .13 

502 

10,13,16; 

.  .  .14 

503 

11,13,16; 

.  .  .15 

504 

9,14,16; . 

.  .16 

505 

10,14,16; 

.  .  .17 

506 

11,14,16; 

.  .  .  18 

507 

9,12,17; . 

.  .19 

508 

10,12,17; 

.  .  .20 

509 

11,12,17; 

.  .  .21 

510 

9,13,17; . 

.  .22 

511 

10,13,17; 

.  .  .23 

512 

11,13,17; 

.  .  .24 

513 

9,14,17; . 

.  .25 

514 

10,14,17; 

.  .  .26 

515 

11,14,17; 

]  ;  ’/.27 

516 

517 

°/,  set  matrix 

of  current  situations 

518 

Situation  =  [ 

0bsAct(16,:)  , ObsPhysDamCl , : )  ;... 

519 

ObsAct (1 , 

:)  , ObsPhysDam (4, : )  ;... 

520 

ObsAct (8 , 

:)  , ObsPhysDam (13, : )  ;... 

521 

ObsAct(ll 

,  :)  , ObsPhysDam  ( 12 ,: )  ;... 

522 

ObsAct (8 , 

:)  , 0bsPhysDam(26, : )  ]; 

523 

524  end 

525 

526 

527  ^stc************************************************************************* 

528  function  [PlotData]  =  DataCalculations(NumResponse,B_n,ResponseMatrix,aPnTrue) 
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529 


530 

531 

532 

533 

534 

535 

536 

537 

538 

539 

540 

541 

542 

543 

544 

545 

546 

547 

548 

549 

550 

551 

552 

553 

554 

555 

556 

557 

558 

559 

560 

561 

562 

563 

564 

565 

566 

567 

568 

569 

570 

571 

572 


l  DataCalcuations  :  function  to  calculate  statistics  for  the  survey 
%  responses 

7. 

°f  Calls:  DTMC,  aPrimeCalc,  GetStats 

7. 

/  LOCAL  VARIABLES  - 

%  N  -  The  numbefr  of  responses  that  stats  are  based  on 

7,  Nrm2  -  2-norm  of  the  difference  between  a’ true  and  a’ 

7o - 

i  initialize  variables 
aPn  =  [1  0  0  0  0]; 
a_n  =  DTMC(aPn) ; 

PlotData  =  []  ; 


/  create  data  points  for  every  10  responses 
for  N  =  10: 10 : NumResponse 

7*  calcuate  the  updated  a’  vector  for  each  situation 
[aPn]  =  aPrimeCalc (ResponseMatrix,a_n,N,B_n) ; 

7*  calculate  stats  of  the  a’  vector 
[Mean, Var ,SD,CV]  =  GetStats (aPn) ; 

Nrm2  =  norm(aPn-aPnTrue) ; 

°/«  concatenate  data 

PlotData  =  [PlotData; [N,aPn, Mean, Var , SD , CV , Nrm2] ]  ; 

end 


end 


7o  \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\ 


function [aP]  =  aPrimeCalc(ResponseMatrix,an,N,B_n) 

/  aPrimeCalc:  function  to  calculate  the  updated  a’  vector  based  on  the 
°/0  situation 

7. 


7,  LOCAL  VARIABLES 
7,  Psim_N 
7.  P_Bn 
7,  Prod_pd 
7,  Sum_pd 
7o  i 

7.  P.aNaN 
7,  Pcond_a 


-  Simulated  probabilities  from  N  responses 

-  Probability  of  the  situation  B_n 

-  product  of  the  physical  damage  probabilities 

-  sum  of  the  physical  damage  products 

-  counter 

-  probabilities  of  actions  that  are  NaN 

-  probabilities  of  actions  conditioned  on  physical 


573 

574 

575 

576 

577 

578 

579 

580 

581 

582 

583 

584 

585 

586 

587 

588 

589 

590 

591 

592 

593 

594 

595 

596 

597 

598 

599 

600 
601 
602 

603 

604 

605 

606 

607 

608 

609 

610 
611 
612 

613 

614 

615 

616 


l  damage 

°/0  aP  -  a’  vector 

y. - 


°/0  get  the  appropriate  probabilities  for  the  calculations 
Psim_N  =  squeeze (sum(ResponseMatrix (1 :N, :,:), 1)/N) ; 

P_Bn  =  [Psim_N(B_n(l) ,:) ;Psim_N(B_n(2) ,:) ; Psim_N(B_n(3) . 

Psim_N(B_n(4) , : ) ;Psim_N(B_n(5) , : ) ;Psim_N(B_n(6) , : )] ; 

Prod_pd  =  squeeze (ResponseMatrix(l :N,B_n (4) . 

. *ResponseMatrix ( 1 : N , B_n (5) 

. *ResponseMatrix ( 1 :  N , B_n (6) , : ) ) ; 

Sum_pd  =  sum(Prod_pd) ; 

/  cacluate  the  conditional  probabilities  of  actions  and  make  sure  all  are 
/  numbers 
for  i  =  1:3 

Pcond_a(i,:)  =  ( sum ( squeeze (ResponseMatrix(l :N,B_n(i) . 

. *Prod_pd) ) . /Sum_pd; 

end 

P_aNaN  =  isnan(Pcond_a) ; 

Pcond_a(P_aNaN)  =  eps; 

°/0  calculate  a’  (define  temp  and  tempsum  are  same  as  in  main) 
temp  =  prod(Pcond_a, 1) . *an. *prod(P_Bn(4 : 6 , :) ,1) ; 
tempSum  =  sum (temp) ; 
aP  =  temp/ tempSum; 

end 


^3|c3|c3|e3|e3|e^c3|c^c^c^e3|e^c3|c9|e^c3|e^e4;3|c^c^e3|e^c3f;3|e^:3|C3fe:4cH<^He3le:lc:f:3lc:4C3f:3ie:4C3le:le3le3le:ic,f:3le:lC3(e3fc^3(c3|c^He:ic,f:3le:lC3ie3le:le,f::ic:lc,f=3fc:4cHcHc:f:3le3ie3lc 

function  [Mean, Var ,SD,CV]  =  GetStats(aP) 

°/0  GetStats:  function  to  calculate  analytical  stats  from  the  a’  vector 

°/. 

/  LOCAL  VARIABLES  - 

y,  SupportX_n  -  The  support  for  BDA  so  mean,  var,  etc  may  be 

°/0  calculated 

#/. - 


y.  calculations 
SupportX_n  =  [12345]; 
Mean  =  aP*SupportX_n’ ; 
Ebda2  =  aP*SupportX_n’ . ~2; 
Var  =  Ebda2  -  Mean''2; 
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SD  =  Var~ . 5 ; 
CV  =  SD/Mean; 

end 


function  N_Plots (PlotData, sit ,NumResponse) 

%  N_Plots:  function  to  plot  statistics  of  the  a’  vector  and  compare  them 
°/0  against  the  true  values 

7. 


7,  LOCAL  VARIABLES 
7,  MnV 
7.  VarV 
7,  tit 

7,  MnMin ,  MnMax 
7,  VarMin,  VarMax 
7,  axl,  ax2,  ax3 
7. - 


-  true  mean  plot  data 

-  true  variance  plot  data 

-  title  of  the  plots 

-  Plot  limits  of  the  mean 

-  plot  limits  of  the  variance 

-  axes  handles 


MnV  =  ones(length(PlotData)-l,l)*PlotData(length(PlotData) ,7) ; 
VarV  =  ones(length(PlotData)-l,l)*PlotData(length(PlotData) ,8) ; 


if  sit  ==  1 

tit  =  ’Situation  \ita’ ; 
MnMin  =  0.5;  MnMax  =  2; 
VarMin  =  0;  VarMax  =  1; 
elseif  sit  ==  2 

tit  =  ’Situation  \itb’; 
MnMin  =  0.5;  MnMax  =  1.5; 
VarMin  =  0;  VarMax  =  0.5; 
elseif  sit  ==  3 

tit  =  ’Situation  \itc’; 
MnMin  =  1.5;  MnMax  =  2.5; 
VarMin  =  0;  VarMax  =  1; 
elseif  sit  ==  4 

tit  =  ’Situation  \itd’ ; 
MnMin  =  2.5;  MnMax  =  3.5; 
VarMin  =  0;  VarMax  =  0.5; 

else 

tit  =  ’Situation  \ite’; 
MnMin  =  3.5;  MnMax  =  5; 
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VarMin  =  0.2;  VarMax  =  1.8; 


662 

663 

664 

665 

666 

667 

668 

669 

670 

671 

672 

673 

674 

675 

676 

677 

678 

679 

680 
681 
682 

683 

684 

685 

686 

687 

688 


end 

PlotData(length(PlotData) ,:)=[]; 
f igure(2*sit) 

subplot (1,2,1) 

plot (PlotData( : , 1) ,PlotData( : ,7) ) 
hold  on 

plot (PlotDataC : , 1) ,MnV, ’-k’ ) 
hold  off 
ax2  =  gca; 

set (get (ax2 , ’XLabel ’ ),’ String’ , ’Number  of  Survey  Responses,  N 
’FontName’ , ’times  new  roman’); 
set (get (ax2 , ’ YLabel ’ ) , ’ String ’ , ’ \mu  Hat ’ , . . . 

’FontName’ , ’times  new  roman’); 
set (ax2, ’FontName’ , ’times  new  roman’,... 

’YTick’ ,0: .1:5, .. . 

’YLim’ , [MnMin  MnMax] , . . . 

’XLim’ , [10  NumResponse] ) 

grid  on 


subplot (1,2, 2) 

plot (PlotData( : , 1) ,PlotData( : ,8) ) 
hold  on 

plot (PlotData( : , 1) ,VarV, ’ -k’ ) 
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698 

699 

700 

701 

702 

703 

704 


hold  Off 
ax3  =  gca; 

set (get (ax3, ’XLabel’) , ’String’ , ’Number  of  Survey  Responses,  N 
’FontName’ , ’times  new  roman’); 
set (get (ax3, ’YLabel’) , ’String’ , ’Variance,  \sigma~2  Hat’,... 

’FontName’ , ’times  new  roman’); 
set (ax3, ’FontName’ , ’times  new  roman’,... 

’YTick’ ,0: .1:2, .. . 

’YLim’ , [VarMin  VarMax] , . . . 

’XLim’  ,  [10  NumResponse]  )°/0 

7. 

grid  on 

propertyeditor ( ’ on ’ ) 
f igure(l+2*sit) 


101 


705  plot  (PlotData(  :  ,  1)  ,PlotData(  :  ,11)  ) 

706  axl  =  gca; 

707  set (get (axl, ’XLabel’) , ’String’ , ’Number  of  Survey  Responses,  N’ ,  .  .  . 

708  ’ Font Name ’, ’times  new  roman’); 

709  set (get (axl, ’YLabel’) , ’String’ ,’ I |a‘  -  ac  Hat  I  I ’ , . . . 

710  ’ Font Name ’, ’times  new  roman’); 

711  set (axl , ’FontName ’, ’times  new  roman’,... 

712  ’ YLim’ , [0  . 2] , . . . 

713  ’XLim’,[10  NumResponse] ) 

714 

715  grid  on 

716 

717  end 

718 

719  ^$He^Hi:He3le3le3le:lcHi::iC3lC3le3fe:t:3lC3lc’t:3le:lc:le3le3le:le9le:4eNC3fe3lcN<3ie$He:lc,ii:9ic:4C3t:3le:t:3lc3ieHi:3le3fe:le3le:4c:f:3le:t:3t::ie:t:3lc$He3le:lc,ii:3lC3lc3lc>le:4e3f:3fe3lcHe:ie3le3le:iC3t: 

720  function  []  =  EngageSequence(ResponseMatrix,Ptrue_All,CondPtrue_Actions) 

721  %  EngageSequence :  function  to  run  through  3  shot  engagement  sequence 

722  °/0 

723  7.  Calls:  DTMC,  aPrimeCalc,  GetStats 

724  l 

725  l  LOCAL  VARIABLES  - 

726  7.  Etas  -  N  values  of  interest 

727  7,  Seq_a,  Seq_pd  -  sequence  of  actions  and  physical  damage 

728  7.  Engagement  Data  -  matrix  of  a’  and  stats  for  output 

729  7o - 

730 

731  7#  ensure  the  porper  number  of  arguments 

732  if  nargin  <  3 

733  for  i  =  1:5 

734  ResponseMatrix( : , : , i)  =  xlsread( ’ Ch4SimData. xls ’ , i) ; 

735  end 

736  Ptrue_All  =  xlsread( ’SurveyProbs .xls ’ , ’p_0’ , ’bll :rl5’ ) ; 

737 

738  CondPtrue_Actions( : , : , 1)  =  xlsread( ’ SurveyProbs . xls ’, ’p_0 ’, ’kl8 : r44’ ) ; 

739  CondPtrue_Actions( : , : ,2)  =  xlsread( ’ SurveyProbs . xls ’, ’p_0 ’, ’k46 : r72 ’) ; 

740  CondPtrue_Actions( : , : ,3)  =  xlsread( ’ SurveyProbs . xls ’, ’p_0 ’, ’k74 : r 100 ’) ; 

741  CondPtrue_Actions( : , : ,4)  =  xlsread( ’ SurveyProbs . xls ’, ’p_0 ’, ’kl02 : r 128 ’) ; 

742  CondPtrue_Actions( : , : ,5)  =  xlsread( ’ SurveyProbs . xls ’, ’p_0 ’, ’kl30 : r 156 ’) ; 

743  end 

744 

745  7#  initialize  variables 

746  aPn  =  [1  0  0  0  0]; 

747  aPnTrue  =  [10000]; 

748  Seq_a  =  [2,4,8;  2,3,7;2,4,8] ; 
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749 

750 

751 

752 

753 

754 

755 

756 

757 
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760 
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764 

765 

766 

767 

768 

769 

770 
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773 

774 
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776 

777 

778 

779 
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782 
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786 
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788 
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Seq_pd  =  [10, 14, 17; 9, 13, 16; 10, 14, 17] ; 

Etas  =  [25,50,100, 250 , 500 , 1000] ; 

'/.  iterate  through  situations 
for  sit  =  1:3 

Engagement  Data  =  []  ; 

1  calculate  a  vector  true  values 
if  sit  >  1 

aPnTrue  =  aPnMatrix (length (aPnMatrix) ,:) ; 

end 

a_nTrue  =  DTMC ( aPnTrue ) ; 

°/«  define  the  situation 

B_n  =  [Seq_a(sit , : ) ,  Seq_pd(sit , : )]  ; 

Bn_pd  =  l+(B_n(4) -9) +3* (B_n(5) -12)+9* (B_n(6) -15) ; 

°/0  get  the  true  probabilities 

P_BnaTrue  =  squeeze ( [CondPtrue_ Act ions (Bn_pd,B_n(l) 
CondPtrue_Actions (Bn_pd,B_n(2) 

CondPtrue_Actions (Bn_pd,B_n(3) ,:)]); 

P_BnTrue  =  [P_BnaTrue ;  ... 

Ptrue_All ( : ,B_n(4) ) ’ ;  ... 

Ptrue_All ( : ,B_n(5)) 5 ; . . . 

Ptrue_All ( : ,B_n(6)) J] ; 

°/*  calculate  a’  true 

temp  =  prod(P_BnTrue , 1) . *a_nTrue; 

tempSum  =  sum (temp) ; 

aPnTrue  =  temp/tempSum; 

°/*  iterate  through  N  of  interest 
for  i  =  1 : length(Etas) 
if  sit  >  1 

aPn  =  aPnMatrix (i ,:) ; 

end 

a_n  =  DTMC(aPn) ; 

°/«  calculate  a}  and  stats  for  n  of  interest 
N  =  Etas(i) ; 

[aPn]  =  aPrimeCalc(ResponseMatrix,a_n,N,B_n) ; 
[Mean,Var ,SD,CV]  =  GetStats (aPn) ; 

Nrm2  =  norm(aPn-aPnTrue) ; 


103 


794  °/«  concatenate  data 

795  Engagement Data  =  [EngagementData;  [N,aPn,Mean,Var,SD,CV,Nrm2]] 

796  end 

797 

798  °/«  calculate  stats  for  a’  true 

799  [Mean,Var ,SD,CV]  =  GetStats (aPnTrue) ; 

800 

801  1  gather  all  data  and  write  to  excel  spreadsheet 

802  EngagementData  =  [EngagementData; [1 , aPnTrue, Mean, Var,SD,CV,0] ]  ; 

803  xlswrite( 5 Ch4EngagementData.xls ’ , EngagementData, sit , ^  b2 , ) ; 

804 

805  aPnMatrix  =  Engagement Data( 2 : 6) ; 

806  end 

807 

808  end 

809  °/oEOF 


Appendix  C.  Simulation  Data  for  Sample  Size  Plots 


Table  C.l:  Sample  Points  of  and  Metrics  for  Situation  a 


N 

ND 

Updated  Distribution  a/(nJ 
M  F  MF 

K 

d 

d2 

||a/(n)  _  £/(n)|| 

10 

1.00000 

0.00000 

0.00000 

0.00000 

0.00000 

1.00000 

0.00000 

0.25743 

50 

0.88289 

0.03237 

0.08474 

0.00000 

0.00000 

1.20184 

0.33058 

0.10968 

100 

0.83915 

0.05715 

0.10370 

0.00000 

0.00000 

1.26455 

0.40197 

0.06094 

150 

0.80473 

0.05210 

0.14317 

0.00000 

0.00000 

1.33845 

0.51025 

0.01602 

200 

0.79899 

0.06039 

0.14061 

0.00000 

0.00000 

1.34162 

0.50614 

0.01280 

250 

0.80286 

0.05300 

0.13897 

0.00516 

0.00000 

1.34644 

0.53534 

0.01219 

300 

0.81353 

0.04812 

0.13310 

0.00525 

0.00000 

1.33006 

0.51881 

0.02439 

350 

0.80166 

0.05821 

0.13135 

0.00717 

0.00161 

1.34886 

0.55220 

0.01435 

400 

0.81159 

0.05568 

0.12343 

0.00791 

0.00140 

1.33186 

0.53282 

0.02651 

450 

0.82666 

0.05009 

0.11356 

0.00772 

0.00197 

1.30824 

0.51028 

0.04458 

500 

0.81776 

0.05093 

0.12194 

0.00727 

0.00210 

1.32501 

0.53204 

0.03241 

550 

0.81790 

0.04834 

0.12549 

0.00626 

0.00201 

1.32613 

0.53242 

0.03106 

600 

0.81661 

0.04646 

0.12804 

0.00677 

0.00213 

1.33135 

0.54378 

0.02917 

650 

0.81392 

0.04893 

0.12748 

0.00705 

0.00262 

1.33553 

0.55167 

0.02653 

700 

0.80695 

0.05426 

0.12848 

0.00769 

0.00262 

1.34477 

0.56043 

0.01963 

750 

0.80686 

0.05160 

0.13138 

0.00744 

0.00271 

1.34754 

0.56674 

0.01811 

800 

0.81159 

0.05127 

0.12700 

0.00724 

0.00291 

1.33860 

0.55624 

0.02445 

850 

0.80723 

0.05193 

0.13080 

0.00736 

0.00268 

1.34632 

0.56429 

0.01868 

900 

0.80638 

0.04897 

0.13508 

0.00704 

0.00254 

1.35039 

0.57046 

0.01674 

950 

0.79842 

0.05251 

0.14099 

0.00555 

0.00252 

1.36124 

0.57629 

0.00746 

1000 

0.79244 

0.05535 

0.14393 

0.00580 

0.00248 

1.37054 

0.58565 

0.00402 

True 

0.79203 

0.05482 

0.14118 

0.00848 

0.00349 

1.37656 

0.60983 
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Figure  C.l:  Situation  a  Moments 


Figure  C.2:  Situation  a  Norm 
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Table  C.2:  Sample  Points  of  a'^  and  Metrics  for  Situation  b 


N 

ND 

Updated  Distribution  a'^ 
M  F  MF 

K 

u 

d2 

||a'(n)  _  a'(n) 

10 

1.00000 

0.00000 

0.00000 

0.00000 

0.00000 

1.00000 

0.00000 

0.04125 

50 

0.99891 

0.00109 

0.00000 

0.00000 

0.00000 

1.00109 

0.00109 

0.04032 

100 

0.98266 

0.00826 

0.00907 

0.00000 

0.00000 

1.02641 

0.04386 

0.02260 

150 

0.96622 

0.00534 

0.02844 

0.00000 

0.00000 

1.06221 

0.11521 

0.00324 

200 

0.97280 

0.00535 

0.02185 

0.00000 

0.00000 

1.04904 

0.09033 

0.00622 

250 

0.97300 

0.00472 

0.02228 

0.00000 

0.00000 

1.04928 

0.09141 

0.00600 

300 

0.97332 

0.00457 

0.02212 

0.00000 

0.00000 

1.04880 

0.09065 

0.00635 

350 

0.97463 

0.00464 

0.02074 

0.00000 

0.00000 

1.04611 

0.08546 

0.00824 

400 

0.97525 

0.00542 

0.01933 

0.00000 

0.00000 

1.04408 

0.08079 

0.00972 

450 

0.96947 

0.00483 

0.02569 

0.00000 

0.00000 

1.05622 

0.10444 

0.00111 

500 

0.97022 

0.00465 

0.02513 

0.00000 

0.00000 

1.05491 

0.10216 

0.00202 

550 

0.96875 

0.00488 

0.02637 

0.00000 

0.00000 

1.05762 

0.10704 

0.00024 

600 

0.97149 

0.00463 

0.02386 

0.00003 

0.00000 

1.05242 

0.09755 

0.00382 

650 

0.97364 

0.00472 

0.02161 

0.00003 

0.00000 

1.04803 

0.08911 

0.00693 

700 

0.97276 

0.00429 

0.02292 

0.00003 

0.00000 

1.05021 

0.09370 

0.00541 

750 

0.97456 

0.00466 

0.02076 

0.00003 

0.00000 

1.04626 

0.08580 

0.00818 

800 

0.97565 

0.00422 

0.02011 

0.00003 

0.00000 

1.04451 

0.08290 

0.00943 

850 

0.97273 

0.00458 

0.02265 

0.00003 

0.00000 

1.04998 

0.09296 

0.00555 

900 

0.97339 

0.00459 

0.02199 

0.00003 

0.00000 

1.04866 

0.09044 

0.00648 

950 

0.97484 

0.00437 

0.02076 

0.00003 

0.00000 

1.04597 

0.08555 

0.00839 

1000 

0.97533 

0.00408 

0.02056 

0.00003 

0.00000 

1.04529 

0.08454 

0.00890 

True 

0.96875 

0.00469 

0.02652 

0.00003 

0.00000 

1.05785 

0.10781 
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Figure  C.3:  Situation  b  Moments 


Figure  C.4:  Situation  b  Norm 
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Table  C.3:  Sample  Points  of  a/(n^  and  Metrics  for  Situation  c 


N 

ND 

Updated  Distribution  a'^ 
M  F  MF 

K 

fi 

d2 

]  a'(n)  -  a'(n)|j 

10 

0.34637 

0.65363 

0.00000 

0.00000 

0.00000 

1.65363 

0.22640 

0.12213 

50 

0.21363 

0.78203 

0.00434 

0.00000 

0.00000 

1.79070 

0.17416 

0.08089 

100 

0.22342 

0.76945 

0.00713 

0.00000 

0.00000 

1.78372 

0.18377 

0.06616 

150 

0.27891 

0.69748 

0.00763 

0.01598 

0.00000 

1.76069 

0.29320 

0.04126 

200 

0.27286 

0.69868 

0.00847 

0.01230 

0.00769 

1.78327 

0.35273 

0.03534 

250 

0.27243 

0.68326 

0.01223 

0.01923 

0.01285 

1.81683 

0.44371 

0.04179 

300 

0.27918 

0.68459 

0.01009 

0.01762 

0.00853 

1.79174 

0.39315 

0.04617 

350 

0.28189 

0.67436 

0.01134 

0.01970 

0.01271 

1.80697 

0.44914 

0.05465 

400 

0.27356 

0.68706 

0.00994 

0.01805 

0.01139 

1.80664 

0.42080 

0.04054 

450 

0.26431 

0.69615 

0.01155 

0.01407 

0.01392 

1.81714 

0.42400 

0.02897 

500 

0.24744 

0.71487 

0.01065 

0.01415 

0.01290 

1.83019 

0.40192 

0.01334 

550 

0.25725 

0.70378 

0.00906 

0.01852 

0.01139 

1.82303 

0.41160 

0.01963 

600 

0.25559 

0.70598 

0.00892 

0.01922 

0.01029 

1.82263 

0.40253 

0.01742 

650 

0.25389 

0.70829 

0.00887 

0.02003 

0.00893 

1.82182 

0.39146 

0.01536 

700 

0.25565 

0.70557 

0.00882 

0.02152 

0.00844 

1.82152 

0.39464 

0.01737 

750 

0.25887 

0.70049 

0.01117 

0.02100 

0.00847 

1.81971 

0.39778 

0.02092 

800 

0.25199 

0.70939 

0.01229 

0.01850 

0.00784 

1.82081 

0.37668 

0.01174 

850 

0.25668 

0.70234 

0.01576 

0.01659 

0.00863 

1.81815 

0.38341 

0.01746 

900 

0.25198 

0.70461 

0.01741 

0.01570 

0.01030 

1.82772 

0.39520 

0.01298 

950 

0.25315 

0.70065 

0.01843 

0.01806 

0.00970 

1.83051 

0.40243 

0.01538 

1000 

0.25319 

0.69944 

0.01738 

0.02082 

0.00918 

1.83336 

0.40867 

0.01601 

True 

0.24252 

0.71112 

0.01860 

0.02068 

0.00708 

1.83869 

0.38156 
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Figure  C.5:  Situation  c  Moments 


Number  of  Survey  Responses,  N 


Figure  C.6:  Situation  c  Norm 
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Table  C.4:  Sample  Points  of  a/(n^  and  Metrics  for  Situation  d 


N 

ND 

Updated  Distribution  a'dd 
M  F  MF 

K 

U 

d2 

]  a'(n)  -  a'Wjj 

10 

0.00000 

0.00000 

0.93185 

0.06815 

0.00000 

3.06815 

0.06350 

0.07507 

50 

0.00000 

0.01401 

0.91413 

0.07186 

0.00000 

3.05785 

0.08253 

0.05937 

100 

0.00758 

0.00639 

0.88333 

0.10269 

0.00000 

3.08113 

0.13284 

0.01631 

150 

0.00479 

0.00599 

0.87692 

0.11230 

0.00000 

3.09672 

0.12811 

0.00976 

200 

0.00315 

0.00501 

0.89344 

0.09840 

0.00000 

3.08709 

0.10843 

0.02680 

250 

0.00526 

0.00371 

0.90327 

0.08361 

0.00414 

3.07765 

0.11890 

0.04223 

300 

0.00447 

0.00372 

0.90384 

0.08513 

0.00284 

3.07814 

0.11198 

0.04188 

350 

0.00901 

0.00637 

0.89129 

0.08920 

0.00414 

3.07310 

0.14281 

0.02967 

400 

0.00878 

0.00823 

0.88588 

0.09225 

0.00486 

3.07619 

0.14925 

0.02387 

450 

0.00701 

0.00688 

0.87253 

0.10985 

0.00374 

3.09644 

0.15042 

0.00396 

500 

0.00674 

0.00622 

0.87242 

0.11084 

0.00378 

3.09870 

0.14938 

0.00372 

550 

0.00710 

0.00593 

0.88227 

0.10054 

0.00415 

3.08871 

0.14360 

0.01536 

600 

0.00665 

0.00709 

0.87284 

0.10868 

0.00473 

3.09775 

0.15175 

0.00380 

650 

0.00575 

0.00670 

0.86063 

0.12071 

0.00621 

3.11492 

0.16206 

0.01463 

700 

0.00552 

0.00615 

0.85742 

0.12466 

0.00624 

3.11995 

0.16349 

0.01967 

750 

0.00617 

0.00604 

0.85546 

0.12611 

0.00622 

3.12017 

0.16729 

0.02203 

800 

0.00580 

0.00570 

0.85742 

0.12528 

0.00581 

3.11959 

0.16310 

0.02012 

850 

0.00561 

0.00546 

0.85137 

0.13111 

0.00645 

3.12732 

0.16860 

0.02848 

900 

0.00522 

0.00539 

0.84874 

0.13450 

0.00615 

3.13098 

0.16822 

0.03277 

950 

0.00509 

0.00496 

0.84582 

0.13680 

0.00733 

3.13632 

0.17287 

0.03644 

1000 

0.00605 

0.00468 

0.84803 

0.13427 

0.00697 

3.13142 

0.17375 

0.03305 

True 

0.00675 

0.00546 

0.87067 

0.11021 

0.00691 

3.10506 

0.15926 
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Figure  C.7:  Situation  d  Moments 


Number  of  Survey  Responses,  N 


Figure  C.8:  Situation  d  Norm 
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Table  C.5:  Sample  Points  of  a'^  and  Metrics  for  Situation  e 


N 

ND 

Updated  Distribution  a'^ 
M  F  MF 

K 

U 

d2 

|  a'(n)  -  a'(n)|j 

10 

0.00000 

0.16416 

0.00000 

0.00000 

0.83584 

4.50751 

1.23492 

0.43192 

50 

0.00000 

0.07093 

0.00000 

0.30101 

0.62806 

4.48620 

0.67540 

0.09430 

100 

0.00000 

0.16643 

0.00000 

0.27351 

0.56006 

4.22720 

1.17415 

0.06679 

150 

0.00000 

0.19009 

0.00000 

0.25672 

0.55319 

4.17300 

1.28363 

0.09481 

200 

0.00000 

0.18495 

0.00000 

0.24670 

0.56836 

4.19846 

1.26876 

0.10005 

250 

0.00000 

0.19229 

0.00000 

0.27936 

0.52835 

4.14376 

1.27685 

0.08585 

300 

0.00000 

0.17693 

0.01290 

0.27574 

0.53443 

4.16766 

1.22695 

0.07518 

350 

0.00000 

0.17401 

0.01138 

0.27405 

0.54057 

4.18118 

1.21514 

0.07275 

400 

0.00000 

0.16971 

0.01087 

0.27694 

0.54248 

4.19220 

1.19524 

0.06739 

450 

0.00000 

0.16524 

0.01020 

0.27626 

0.54830 

4.20761 

1.17637 

0.06412 

500 

0.00000 

0.16126 

0.00703 

0.26923 

0.56248 

4.23293 

1.16031 

0.06739 

550 

0.00000 

0.15074 

0.00745 

0.27497 

0.56685 

4.25793 

1.11071 

0.05798 

600 

0.00000 

0.15590 

0.00673 

0.29146 

0.54591 

4.22738 

1.12455 

0.04675 

650 

0.00000 

0.14393 

0.00754 

0.30937 

0.53916 

4.24377 

1.06300 

0.02924 

700 

0.00000 

0.14338 

0.00768 

0.31642 

0.53252 

4.23808 

1.05706 

0.03012 

750 

0.00000 

0.14233 

0.00757 

0.32659 

0.52351 

4.23128 

1.04691 

0.03560 

800 

0.00000 

0.15099 

0.00747 

0.33144 

0.51009 

4.20063 

1.08129 

0.05192 

850 

0.00000 

0.15081 

0.00737 

0.32116 

0.52066 

4.21168 

1.08646 

0.04287 

900 

0.00000 

0.14057 

0.00418 

0.33116 

0.52409 

4.23876 

1.03356 

0.03455 

950 

0.00000 

0.13527 

0.00410 

0.34215 

0.51847 

4.24383 

1.00420 

0.04093 

1000 

0.00000 

0.12981 

0.00291 

0.33089 

0.53639 

4.27386 

0.98354 

0.01922 

True 

0.00107 

0.12224 

0.00173 

0.32290 

0.55206 

4.30264 

0.96079 
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Figure  C.9:  Situation  e  Moments 


Figure  C.10:  Situation  e  Norm 
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